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ABSTRACT » 
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infant/toddler component of CFRP and its impact on children 
approximately a year and a half after they enter \the program. A brief 
summary of the CFRP evaluation design and preliminary findings 
presented in previous evaluation reports is given in chapter 1. 
Chapter 2 provides a description of the infant/toddler component of 
the 11 CFRP's in terms of center-based activities, frequency and 
leve l of pa rticipation, and approaches used in ]6oth parent education 

Tnf ant/toddler sessions. Chapter 3 examines CFRP impact on the 
development of the infants and toddlers invol|ved; differences in 
means between CFRP children and those in the Icontrol/comparison group 
on the Bayley Scales of Infant Development (BpiD) are reported in the 
attempt to identify major program impact. An executive; summary of the 
study's findings is presented in chapter 4. Fo^ur appendices, 
accounting for approximately one half of 
included: the first reviews issues related 

child assessment data reported, and the second discusses the 
rationale for the statistical tests chosen. A siep-by-step 
description of the analysis of program impact reported in chapter 3 
is presented in the third appendix, while the fourth describes 
analysis of the relationship between program participation a«dBSlD 
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FOREWORD 

In 1973, the Administration for Children, Youth 
and Families (ACYF) initiajted the Child- and Family Resource 
Program (CFRP) as part of ■ the Head Start Improvement and 
Innovation planning effort- CFRP was funded as a demon- 
stration program with' the intent of developing models 
for providing services to low-income families with young 
children—models which could ^e adapted by different commu- 
nities serving different populations. There are eleven CFR 
programs across the country, one in each of the ten HHS 
(formerly HEW) regions and one representing the Indian and 
Migrant Division. Each program receives approximately^ 
$155; 000-$l70, 000 per year to serve a minimum of 80 fami- 
lies. 



CFRP is a family-oriented child development 
program which provides support services crucial for the 
sustained healthy growth and development of families who 
Jiave children from the prenatal period through age eight. 
It promotes child development and meets children's needs 
by working through the family^^as a unit and provides con- 
tinuity in serving children during the major stages of their 
early development. CFRP services are offered within the 
context of three major program components — inf ant-rtoddler , 
Head Start, and preschool-school linkage. Each is intended 
td serve families with children in a specific age group; all 
three taken together are intended to provide continuity — 
especiall y developmental - and educational continuity — across 
the period of a child's life from before birth to the 
primary grades iri school. 



Another distinctive feature of CFRP is its emphasis 
on a comprehensive assessment of each family's strengths and 
needs and the development with the family of an individualized 
plan for services to be. obtained through CFRP. Families 
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enrolled in CFRP receive the \same comprehensive services 
that are offered by Head Start and additional services 
tailored to- the needs of :each \faniily. At the same time, 
CFRP works to reduce fragmentation and gaps Un the delivery 
of services, by existing community programs and agencies.. 

In October 1977, the Administration for Children, 
Youth and Families "funded a longitudinal evaluation to. 
determine the effectiveness of the Child and Family Resource 
Program. "the evaluation is designed to address three major 
questions: 

# What is the nature and extent of services 
that Should be provided to families and 
children in order to meet their needs, 
enhance their strengths and foster 
independence? 

# V^That are effective processes for the , 
provision of these services? 

# What can be learned about the develop- 
mental processes* of families and, how they 
relate to the developmental processes of 
children? 

,\ 

The current evaluation of CFRP was^ preceded by two 
other studies of the program, both also funded by ACYF. The 
first, conducted by Huron Institute in 1974-75, wa^ an effort 
to determine the feasibility of a summative evaluation of 
CFRP. A formative evaluation of CFRP was also undertaken in 
1974-75, by Development Associates Inc. A. follow-up study 
was conducted by the same contractor in 1975-77. 

This is the fifth in a series of CFRP evalua- 
tion reports. The first report presented the overall study^ 
design. Study implementation and the collection of base- • 
line data on evaluation families were the focus of the 
second report. The third reiport consisted , of three volumes: 
Volume I documented the first six months of the study and 
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examined initial program impact on families; descriptive 
information about CFRP operations at the six evaluation 
sites was presented in Volume II; .the third volume was a 
summary of the findings presented in the first two. The 
fou'rth report presented descriptive profiles of all eleven 
CFRPs. 

This fifth report focuses on the infant-toddler 
component of CFRP and its impact on children approximately 
a year to a year and a half after they entered the program. 
Chapter 1 brief ly Summarizes the CFRP evaluation design and 
preliininary findings that were presented in previous evaluation 
reports. A description of the infant-toddler component in 
the' eleven CFRPs. is presented in Chapter ^2. To the extent 
possible, we have attempted to identify program models for 



the delivery of services to fami 
infantr-toddler age range. We al 



lies with children in the 
so report on the frequency 



of family participation in various infant-toddler activities 
at five of the, six impact study ^sites through March 1980. 
These data are presented in an attempt to determine the 
extent to which' the infant-toddler component as now operated 
is conducive to achieving the objectives of CFRP as articu- 
lated in the national Guidelines . • 

Chapter 3 examines CFRP impact on the development 
. of the infants and toddlers who are the focus of this 
longitudinal evaluation. Differences in means between CFRP 
children and those in the control/comparison group on the 
Bayley Scales of Infant Development are tested in an attempt 
to identify any major program impact. In this chapter, we 
also explore the relationship between level of participation 
in CFRP and positive outcomes for children. An executive 
summary of finding's is presented in Chapter^ 4. 

* 

There are four appendices to this report. Appendix 
A reviews issues related to the quality of the child assessment 
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data reported here. Appendix B contains" a discussion 
of analytic models often used in testing program impacts, 
and the rationale for the st^atistical tests chosen. Appen- 
dix C provides^ a step-by-step description of the analyses of 
program impact reported/an Chapter 3. A description of tl)e 
analyses of the^xeiar^ionship between program participation 
and Bayley scores is presented in Appendix D. 
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Chapter 1 



EVALUATION OVERVIEW 



/ The CFRP evaluation, funded in October 1977 

by the Adkninistration for Children, Youth and Families 
(ACYF), sfeeks to provide detailed information about the 
ef fectivjeness of CFRP as a whole, of individual program.s 
.and of particular program elements or configurations of 
elements. ,,Such information can aid ACYF in making de- 
cisions about expansion of the program and/or dissemina- 
tion of its most important and effective features. 



The first phase of the CFRP evaluation was 
devoted to redesign, start-up of the study, and collec- 
tion of baseline data. In Phase II, the evaluation 
examined program impact on families after six months 
in the program, as well as CFRP treatment and proceisses 
used to deliver service^ to families. The major focus of 
the third phase ofx the^^valuation i-s on program itppact after 
families have participated in CFRP for a year and a half. 

The initial design for the CFRP evaluation 
consisted of three distinct but interrelated components 
which address the following objectives: 

• -dies ctipt ion CFRPs and their operations; 



• 
• 



identif icationXpf program models; 

Linking of f amily^^utcomes to particular 
aspects of CFRP .treatment (characteristics 
of staff and programr^^d to 'r^amily 
characteristics; and 



linking of family outcomes tb^ 
or nonparticipation in CFRP . 



^participation 



The three component studie^^x^prograni/ impact, and process/ , 
treatment — are complementary, ways of viewing the effects and 
effectiveness of CFRP . V 

The program study is designed for the purpose of 
developing a comprehensive picture of the operations of CFR 
programs. Information collected during site visits and in 
interviews with program staff is used to develop profiles of 
program implementation and to establish a descriptive -context 
for the statistical and analytic findings of other components 
of the evaluation. Site visits took place at six of the 
eleven CFRPs in fall 1978, spring 1979, and spring 1980: 
Jackson, MI; Las Vegas, NV; New Haven ^ CT; Oklahoma City, 
OK; St. Petersburg, FL; and Salem, OR.^. These six programs 
were not randomly selected; they were chosen pn the basis of 
their ability to recruit the requisite number of families 
* f or the impact study. Brief interviews were conducted in 
spring 1980 with staff from th^ five non- impact study CFRPs 
to obtain descriptive infdrmation about the operations of 
these programs. 

! 

The impact, study is designed to determine the 
effects of CFRP services on families by comparing CFRP 
families with a group not enrolled .in the program. At 
the six sites listed above, families^ entered the evalua- 
tion when they had a child less than one year old and were 
randomly assigned either to CFRP or to a control/comparison 
group. At entry into the evaluation/, there were an average 
of 39 CFRP and 38 .control/comparisori families per site. 
These families will be- followed unt/il the : focal child has 
completed-^ least one year of elementary school (1985).^ 



^^ ^ ^ .^The impact study focuses pn five outcome domains 
likely to be affected by family participation in CFRP: 



m 



family circumstances (e.g., employment/ 
education) ; 

maternal and child health; 



o parent-chi^^d- relationship and interaction; 

• child development and achievement; and 

• capacity fpr ind,ependence (use of community 
resources, ' locus of control and 'coping 
strategies, affiliation with family and 
social networks). 

This report focuses on the domain of -child develop- 
ment arid achievement. Data concerning child development 
were obtained for the, first time in fall/winter of 1979-80. 
Program impact on families in the other four outcome domains 
was examined in the Phase II Report. There was little 
evidence that the program had had a positive impact on the 
sample families in these domains after six months of partici- 
pation, '^thig^mayb^ to the fact that_.such a 
>■ ' period is too short for impact to become apparent. For 
example, changes in family circumstances or capacity for 
V ' independence may not become evident until the family has 
been involved in the program for a longer period of time. 
However ,^ resuats --o£^--a^piJjD±^ati^ of paren t -child interaction 
conducted in spring 1979 did provide, preliminary evidence of 
program impact in this area. Specifically, CFRP mothers had 
more frequent interactions with their children than was the 
^ case for mothers in the control/comparison group — although 
these findings were largely site-specific* Positive changes 
in parent-child ^interaction are expected to influence the 
development of the child. 

To date, impact study data have been obtained at ^ 
four time points: fall 1978 (baseline), spring 1979 (six 
months after the families entered the evaluation), fall/winter 
1979^80 (after approximately one year of program participation) 

*Re^ults of the parent-chi'ld interaction observation pilot 
study are reported in Appendix E of the Phase II Report , 
Volume It Research Report , .February 25, 1980. 



ana spring 1980 (after about 18 months of trea^traent). Sample 
sizes at the first three data collection points (not including 
spring 1980) are noted in Table 1-^1. As is evident from the 
table, sample attrition was most severe during the first six 
months of the CFRP Evaluation. By fall/winter 1979-80 the 
attrition rate appeared to have leveled off considerably. 
This was partly due to a decision to retain families in the 
evaluation who decided they ^no longer wanted to participate 
in CFRP. There were 13 former CFRP families in the fall/winter 
sample (8 in Jackson and 5 in.St. Petersburg). 



Table 1-1 
Impact Study . Sample Sizes 



Fall 1978 
CFRP Control 



Spring 1979 
CFRP Control 



Fall/winter 

1979-80 \ 
CFRP Control 



Jackson 


40 


24 


31 


> 20 


30 


20 


Las Vegas 


42 


43 


32 


33 


35 


29 . 


New Haven 


36 


20 


28 


18 


26 


14 


Oklahoma City 


39 


49 


32 


45 


28 


43 


St. Petersburg 


40 


43 


34 


40 


34 


38 


Salem 


39 


51 


31 


42 


34 


40 


Total' 


236 


230 


188 


198 


187 


184 


Site Average 


39 


38 


31 


33 ' 


31 


31 


Attrition 






2J.-^ 


1 3% 




6% 



The process/treatment study focuses on the CFRP 
families who participate in the impact study at the six 
sites. This study is designed to explore^ relationships 
among characteristics of families and staff, interactions 
between staff and families, servit^es provided, and program 
impact. Data were collected in fall 1978, spring 1979, aVt/ 



spring 1980 through interviews with staff and families. 
In addition, ongoing data collection systems are being 



/ 



maintained for data concerning family participation in th^ 
program, family goals, and referrals for services. 



4 



A fourth component has^ been added to the evalua- 
tion in Phase III. The ethnographic study is designed to 
broaden our understanding of how CFRP works with families ^ 
and functions as a child development • and family support 
program. Data will be gathered through a series of in-depth 
interviews and observations to be conducted beginning in 
fall 1980. 



Chapter 2 



THE INFANT-TODDLER COMPONENT 



I The primary objective of the inf ant-toddler corri-r 
Iponent of CFRP^ is "to assist parents to promote 
I the, total ( emotional , cognitive , 1 anguage , and 
I physical) development of infants and toddlers 
[through age three-" The eleven CFRPs attempt to 
[achieve this objective through home visits and 
1 center-based activities — parent education sessions 
land infant-toddler sessions* However, at most 
jsi^s/ parent educationr>sessions offer little or 
I no afreet demonstration of techniques for working 
I with child,ren, and home visits tend to focus on 
Ihelping parents meet specific needs rather jthan 
Ion enhancing the development of the child. More- 
lover^r levels of participatdon in both center-based 
land home-based program activities are low in many 
I instances* Thus there, is reason to doubt that 
I the infant- toddler component, as currently- imple- 
Imented, ^^can have significant positive effects on 
I children ' s development . \ 



According to the national CFRP Guidelines , the 
objective of CFRP^s infant- toddler component is to enhance 
„t:he--to-ta4— deA^elopmep^--^^ t o ddlers through a g e 

three. ^ There are essentially three different approaches, 
that could be used to attain this goal: (1) direct inter- 
vention with children; .(2.) parent education to assist 
parents in their role as primary educators of their own 
children; and (3) a combination of the two. The second- 
approach is advocated in the Guidelines ; it is by working 
through parents and the family as a unit that CFRP expects 
to influence the development of* children. Numerous research 
studies support this focus. The evidence indicates that, 
parent education not only can be an effective strategy in 
promoting child development, but may be a necessary step if 
any lasting improvement in the child *s functioning is to be 



attained. Thus involvement of the child's parents as active 
participants appears to be critical to the success of a child 
development program such as CFRP. 

In practice, the parent education approach to 
providing inf ant-toddler services has been adopted generally 
by local CFRPs, with some secondary emphasis on direct inter - 
vention . Parent education sessions and home visits are the 
principal parent-focused activities at the eleven CFRPs . In 
addition, most of the programs offer some form of group 
activity for children (/"infant-toddler sessions"). 

^ This chapter presents descriptive information on \ 
the infant-toddler component of CFRP. Section 2.1 deals 
with, ceinter-based activities, including frequencies-, levels 
of participation, and approaches used in both parent educa- 
tion sessions and infant-toddler sessions. Section 2.2 pre- i 
sents parallel information for home visits. Section 2.3 
summarizes data on participation for all program activities 
combined . 

2.1 Center-Based Activities ^ 

2*1.1 Frequencies and Participation Rates 

While most CFRPs offer two to four center session? 
per month (Table 2-1), most parents actually attend much lefes 
frequently.' Family participation in center sessions is Viewed, 
by all programs as "less than optimal . " For five of the six 
evaluation programs*, this means that only \39 percent of the 
study families were involved in center sessions an average | of 
once or moire per quarter since they enrolled in CFRP. Parti- 
cipation varied from site to site, however, as noted in Table 
2-2. Participation was particularly problematic isn Oklahoma 
City and Las Vegas.. Problems with attendance in Oklahoma City 

*The '^sixth. New Haven, ,was excluded from participation 
analyses due to a high incidence of missing .data. 



Bismarck 
Gering 
Jackson 
Las Vegas 
Modesto 
New fiaven 
Oklahoma City 
Poughkeepsie 
St. Petersburg 
Salem 

Schuylkill Haven 



Table 2-1 

Frequency of Center-Based Activities 

2 times/ .3 times/ 2 times/ 

week Weekly month , 

X 



month Monthly 



X 



X 
X 



X 
X 



X 
X 



^Frequency of parent education sessions varies depending 
on the season. At harvest time, sessions occur .twice, a 
month; weekly sessions are offered in the off sea^son. 

^St. Petersburg's monthly center • sessions are supplemented, 
by wejekly study groups. 



Table 2-2 

Participation in Center- Sessions 
(percent of families) 



Okla- St. 



At leafet once 
per \quarter 

Less than once 
per quarter 



Jackson 
' 38 



61 



39 



Las 


• lioma 


Peters- 






Vegas 


City 


burg • 


Salem 


Overall 


36 


30 


34 


39 


177 


17 


13 


50 


49 


' . 39 


83 


. 87 


50 


51 • 


, 61 



/ 



^Figures \are based on the ongoing ^ record-keeping system 
for CFRP\ families in the. impact study. (See chapter 
note for an explanation of techniques used to compute 
participation rates . ) 



undoubtedly were due to the fact that center sessions were 

not offered for some time during the first year and a half 

. \ ■■■ ■ . 

after the families entered CPRP; centejr-based activities were 

resumed at" this site in winter of 1980. In La^ Vegas, pro- ' 
blems with center attendance can be attributed to character- 
istics of the families that were selected for the evaluation; 
many of the teenage mothers in the/ evaluation sample attend 
school during the day and are una^Dle to participate, in daj^- 
time center activities. Center participation was less pro- 
blematic iii Jackson,. St. Petersburg, and Salem, where 49 to 
61 percent of the families attended sessions regularly. 

/ ■ . 

Table 2-3 shows pajeticipation m center sessions 
for only those evaluation families who attended regularly," 
i.e. at least "once per quarter on average (henceforth termed 
"center" families). Most,' of these families ('68%) attended 
one to three sessions pet quarter; very few participated in 
all sessions that were^bf f ered by their local pirograms. 
Families who came to the center regularly attended an average 
of 3.4 sessions per (Quarter. Mean attendance rates ranged 
from a low pf 2.2 s^essions in Las Vegas to a high of 4.3 in 
St. Petersburg. 0t. Petersburg rates are high partly be- 
cause the data include weekly study groups as well as monthly 
parent education sessions.) Participation of other families- 
in the evaluation sample (those attending less than once per 
quarter) averaged .24 sessions per quarter-, or one center 
session every 12 months. 

Tables 2-4 to 2-7 break out. participation rates for 
infant- toddler and parent education sessions, respectively, 
for the sample o^ families whose children were tested in 
connection with the child impact study. (In Chapter 3, child 
ren's test performance is examined in relation to attendance 
at these two types of center sessions, taken separately . ) 
•The data for this somewhat different sample (see table notes 
for explanations of the differences) are largely consistent 



Table 2-3 

. . a 
Center Participation per Quarter 

(percent of ."center" families) 

Okla- St. 





Jackson 


Las 
Veqas 


homa 
City 


Peters- 
burg 


S a leiti 


Number of sessiorls 
offered per month 


2 


2 


2 , 


-1 


4 


N of families 


23 


6 


. 4 


17 


19 


Number of sessions 

attended per quarter 










/ 1 ' • 


35 


33 


75 


6 


21 


2 ; ■ •; . 


26 


50 


0 


35 


16 


3 


17 


17 


0 


18 


16. 


4 or ' more 


21 




25 


42 


47 


Mean number of 

se s s ions attended 
per quarter 


2.8 


2.2 


3 . 2 


4.3 


3.8 



69 



26 
26 
16 
32 



3:. 4 



^See' chapter note for an explanation of computational procedures 
used to derive these percentages. New Haven is omitted from the 
table because of ' its high incidence of missing data. Sample in- 
cl^Jdes all families who ^participated in center sessions ' at least 
once per quarter and who remain active participants in CFRP. 

# ■ • . 

Data for St. Petersburg include weekly study groups as W^;>11 as 

monthly parerlt education sessions. 



\ 



N 



Table 2-4 

Participation in, Pajrent Education Sessions 
(percent of families) 

Okla- St. 
Las homa Peters- 
Jackson Vegas City burg . 

24 37 27 



Less than once 
per quarter 

Once or more 
per quarter 



33 
67 



92 
8 



93 
8 



25 
56 

44 



Salem . 
28 

54: 

46 



^Sample includes families whose children were tested in the 
child impact study. Some of these families have since ter- 
minated participation in CFRP. 



id- 



22 




^ Table 2-5 

Participation per Quarter in Parent Education Sessions 
(percent of center" families)' 



Jackson 



Las 
Vegas 



Okla- 
homa 



St. 

Peters- 
burg 



S^lem Overall 





16 


3 


2 


11 


13 


45 


Number of sessions 
attended per quarter 














•1 


31 


100 


.50 


55 


38 


44 


'^i ^. 


25 


0 


0 


27 


31 


24 


3 ' 


19 


0 


0 


0 


15 


11 


4 or more 

t ■ 


25 


0 


50 


18 


15 


20 


Mean nuraber of 
sessions^ attended . 
per quarter 


2 .9 


1.0 


3.3 


2.3 


.2.5 


2.6 



^'Sample includes families v/hose children weife tested in the child^ 
impact study. Some of these families have since terminated parti 
cipation in CFRP. * ^ 



Table 2-6 . . 

Participation in Infant-Toddler Sessions 
(percent of families) 

Okla- St. 
Las homa Peters- 
Jackson . Vegas City burg Salem Overall 

24 37 27 25 28 141 



Less than once ^^^^^-^^ ^^^^ -^ , ■ ' 

per quarter . 33 • | 84 96 56 50 66 

Once or more ^ 

per quarter 67 • 16 4 44 ' 50 34 



^Sample includes . f^iliejs Whose children were tested in the child^ 
impact study. Some of jthese families have since terminated parti 
cipation in CPRP. 



11 



.23 



Table 2-7^ 

.Participation per -Quarter in Infant-Toddler Sessions 
(percent .-of "center" families) 



Number of sessions 
attended per 
quarter 



JacKson 
16 



Las 
Vegas 

i . 6 



Okla- 
homa 
City 

1 



St. 

Peters- 
burg 

11 



Salem Overall 



14 



48 



1 


.31 


100 


0 


' 27 


50 


44 


2 ■ 


19 




0 


36 


29 


23 


3 


25 


0 


100 


0 


21 


.17 


4 or more 


25 


0 


0 


36 


0 


17 


Mean number of 
sessions attended 
per quarter 


3.1 


1.0 


3.0 


2.8 


2.2 


2.3 



^Sample includes families whose children were ,t^stqd in the 
child impact study. Some of these families have since ter- 
minated participation in CFRP. 



with the data in Table 2-3: For each type of center session, 
there are a significant number of nonpc.vticipating families, 
especially in Las Vegas and Oklahoma City. Among those who 
do participate, frequencies are typically in the range of one 
.to three times per quarter, although ythere are a few families 
at certain sites who participate much more ifrequently. 

CFRP staff attribute occasional nonparticipation 
mo 5 1 ly to i 1 Ine s sT^'crl ses ~ or. eme~^gencies that prevent 
parents from attending. However/ chronic nonparticipation 
on the part of some families represents a problem for all 
programs. Sdme mothers simpl/ do not wish to join a group 
or do.^not believe they will iSenefit from being involved. 
Others consider it "too risky" to attend — in the sense of 
feeling vulnerable or 'deficient — or lack the necessary 
support from husband pr .family. 



As already implied, participation is particularly- 
problematic for mothers who are employed or attend sciiool 
during the day, when center sessions typically take place. 
Gering and St. Petersburg are th\e only programs that conduct 
evening sessions for parents on a regular basis-~once a 
month — to accommodate the working or in-~school mother. In 
other^ programs , evening sessions are a rare occurrence.. 
Some programs that have tried them found that participation^ 
did not increaifee; parents are simply too tired after a full 



day of work or 
even more time 



schoo,l to attend, or do p^t-Woffl 
away from bejing with their children. 



A. variety of approaches are used by local programs 
in an attempt to increase participation in center sessions . 
All CFRPs except orte provide transportation for parent's^ who 
could otherwise not attend. Several programs hold theirx^ 
center sessions in more than one location to make them more * ' 
accessible. Others offer some sort of tangible incentive or 
have established policies concerning minimum participation 
in center-based activities; these policies appear to h'qive a 
positive influence on attendance rates. 

2 .1 . 2 Center Session Content and Models 

Parent education sessions are intended to provide 
families with a basic knowledge^ of child growth and develop^ 
ment and to assist them in developing more of f ective parenting 
skills. Infani^-toddler sessions are intended to provide 
children with a group experience and give them an opportunity 
to learn; to share and get along with others. In addition, 
some" programs emphasize acquisition of skills, such as 
\ language, cognitive, motor, social-emotional, and self-help . * 



*In a few CFRPs, ^.essions for infants and toddlers are guided 
\by the same curriculum as that used for home visits. It is 
not uncommon for home visit staff to participate in these 
sessions. Each program has developed special mechanisms, to 
ensure some level of continuity between center- and home- 
based activities, either in the form of records or periodic 
meetings with appropriate staff, although. the degree of coor- 
dination is not 'high in most cases. (See the next section 
for dii^cussion of the content of the home visit curriculum. 
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All ^re .designed to enhance the child's overall development , 
and these approaches prepare the child for entry into Head 
Start. ' iliidividualization of kctivities to meet the specific 
needs -of each child is quite limited in most programs.) 

Despite these indications of a concern for the 
education and development of thc\ child, however, center-rbased 
sessions at most sites are not o3^,ganized in a fashion that 
is liXely^to maximize their developmental effects. Two 
models for integrating the parent ^education and infant-toddler 
portions of center-based sessions, ^re currently in operation 
at the eleven CFRPs: 

9 The Parent-Child Interaction kodel provides 

extensive opportunity £br involvement of parents 
with their own children\at the center. Center 
sessions are designed to help parents acquire 
effective child care tecifiniques' an'd to teach 
them developmental activities that are,, appro- 
priate' to the child's ne4ds. Classroom staff 
assist parents in this t^sk and provide feed- 
back on parent-child interactions. The group 
.discussions that^ follow fe)cus on topics related 
to child development or child-rearing practices. 

• In the Separate Parent-Chi|.d Session Model , 
parent education focuses aJ^most entirely on 
parents, away from their children. Children • 
are cared for in an inf antHtoddler room while 
parents attend parent education sessions. 
There is little or no opportunity for parents 
to interact with their jchildren at the center. 

The second model is likely to b|p somewhat less 
effective, because it relies mostly on lectures and other 
didactic approaches as methods of parent i+raining. , There is 
little evidence that simply providing infcjrmation to parents 
will in itself lead to significant change lin parental behavior 
or skills.* Observation of modeled behavior, which is regarded 



*Br6nfenbrenner, U. ^ Is Early Education Effective ? Washington 
dIc. DHEW Publication No. OHD 74-75, 1974. 



as a more effective learning tool, is used extensively in the 
Parent-Child' Interaction Model. This model is in place at only 
three programs — Bismarck, Gering, and New Haven; all other pro- 
grams conduct separate sessions for parent and child. (Salem 
is a partial exception, in that it has offered opportunities 
for parent-child interaction to selected families with toddlers 
who have speciaj^ needs ; its regular parent education program 
involves separate sessions.) . - 

2 . 2 Home Visits 

2.2.1 Scheduled and Actual Frequencies 

Center sessions are not the only mechanism for edu- 
cating parents of infants and toddlers. A regular home visit- 

-xng^r ogram e arn— arlrso— h^lp-^arreBtrS--tQ--^ta:^engt.^^ c hilA-^ 

rearing skills and increase their knowledge about child de- 
velopment. Home visits are an integral part of the infant- 
toddler component at all eleven programs . The potential 
importance of the home visits is underscored by indications, 
discussed in the previous section, that center-based parent 
education may be less than optimal as currently conducted. 

There is some question,' howeve^r, whether visits 
occur with sufficient frequency to carry out an effective 
parent education program in the home. (The importance of 
frequent visits was shown in the eva,luation of the Home 
Start Demonstration Program, which found a strong relation- 
ship between visit frequency and school' readiness and language 
development scores of preschool children.*) Scheduled fre- 
quencies of CFRP home visits range from one to four times a 
month (Table 2-8). However, it is evident from program records 
and discussions with CFRP staff that at some sites, at least, 
home visits occur less frequently than the schedule called 
for in local program plans (Table 2-9). 

*Love, J.M., Nauta, M.J,, Coelen, C.G., et al . National Home 
Start Evaluation: Final Report — Findings and Implications , 
High/ Scope Educational Research Foundation, Michigan, and Abt 
Associates Inc., Massachusetts, 1976. 





Table 2-8 




1 


Home Visit Frequency 






3 times/ 2 times/ ^ 




1 


month month Monthly Varied 






Bismarck ^ 






Gering x 






Jackson x . * 




1 


Las Vegas x 




■ 


Modesto . ' ^ 






New Haven ^ ■ .. 




1 


Oklahoma City x 






Poughkeepsie x 






St. Petersburg x 






Salem ^ 




1 


Schuylkill Haven x 




1 . 


^In these programs,* home visit frequency varies depending 






on family need and interest . 




1 


In addition to showing the relationship between 






scheduled and actual home visits, Table 2-9 illustrates 




1 


another important point — namely, that home visits and center 






attendance go hand in hand, rather than being alternative or 




1 


complementary ways in which families take part in CFRP. 






Families who participate in center sessions less than once 






per quarter ("non-center" families) receive considerably fewer 






home visits than families who come to the center regularly. 




■ 


Families in the latter group ("center" families) Vere visited 




■ 


about two, times per month on the average, while "non-center" 






families were seen only once a month. Only in Las Vegas were 






the two groups of families involved in home visits at approxi- 






mately the same rate. Across all sites, the correlation be- 




1 


tween home visit rate and center participation was .49. 






•These differences in home visiting rates for the 






two groups are somewhat surprising. One might have expected - 




1 


home visits to oc'tiur with greater intensity . with families who 




Ierlc 







Tahle 2-9 

Home Visit Participation per Quarter 
(percent of families)* 



Home visits 
offered per month 



Jacikson Las Vegas Oklahoma City St. Petersburg Salem Overall 

Center Non-Center Center Non-Center Center Non-Center Center Non-Center Center Non-Center Center Non-Center 



varied 



varied 



N of families 



23 



15 



30 



26 



17 



15 



19 



20 



69 



108 



Nunt>er of heme visits 
per quarter 

1 0 20 

2 0 13 

3 4 27 

4 13 7 

5 4 20 

6 17 0 

7 17 7 

8 17 0 . 

9 _9 0** 

10 9 

11 4 

12 4 



7 
0 
0 



Mean number of heme 

visits per queurter 7.61 



S.D. 



2.36 



4.04 



2.51 



67 
17 
0 
17 
0 
0 
0 
0 
0 
0 
0 
0 



2.33 
1.15 



30 
47 
10 
13 

0 

0 

0** 



0 
0 
0 
0 
0 



2.41 



1.01 



25 
50 
25 
0 
0 
0 
0 
0 
0 
0 
0 
0 



73 
15 

12** 



0 
0 
0 
0 
0 
0 
0 
0 
0 



17 
12 
18 
6 
18 
12 
12 
6 
6 
0 
0 
0 



27 
7 
13 
33 
7 
7 
0 
0 
7 
0 
0 
0 



5 
IX 
16 
16 
26 
16 
5 
5 

0^ 
0 
0 



10 
5 

40 

25 
0 
5 
5 

10 
0 
0 
0 
0 



2.37 
.88 



1.68 



.70 



5.96 



2.38 



3.61 



2.01 



5-80 
1.77 



4.19 
1.89 



7 
9 
9 
14 
7 
17 
13 
10 
6 

4 
1 

1 



5.94 



2.63 



36 
20 
19 
14* 
4 
2 
2 
2 
1 
1 
0 
0 



2.98 
1.86 



*Center families are those Who participated In center sessions at least once per quarter; non-center families attended less frequently. 
**The underline derates ^e nuriber of home visits that are supposed to take place according to individual program schedules. 
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never come to. the center or come only occasionally, in order 
to ensure that all families receive all of the benefits offere 
by CFRP. Instead, it appears that^ non-center families are 
simply less committed to CFRP than other families served by 
the program, perhaps due to a lack of interest or motivation 
to participate or, in the opinion of parents, less need for 
CFRP services. 

2.2.2 Home Visit Con tent and Models 

— — - ' , , — 

In geueral, home visits do not represent a contin- 
uation of the curriculum or activities presented at center- 
based parent education sessions. At most sites, there is no 
explicit attempt to follow up on center activities in the 
home. One reason is that, while an effort is made to adapt 
center sessions to the' needs of those present, they are 
nevertheless group sessions. Home visits, on the other 
hand^can be highly individualized. 
. - < 

Discussions with staff- suggest that child-related 
parent education activities provided in the home typically 
involve helping parents to use elements in the child's 
environment as teaching tools and to turn everyday experi- 
ences into constructive learning situations. Parents are 
reminded about the teaching potential of all household tasks 
and the many objects in the home that can be used as instruc- 
tional materials. In 5ome programs, staff bring specific 
activities into the home to involve both parent. a,nd Child.. 
Usually the activity is preceded by an explanation of its 
importance and how it fits into the overall development of 
the child. An attempt is made not only to demonstrate 
activities to the parent, but to get her actively involved 
in working with the child- Frequently, a different set of 
activities is selected or planned for each family to ensure 
that they are appropriate to meet specific parent or child 
needs. 
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Despite the potential value of these activities 
for child development,, it is not clear that they receive 
adequate emphasis in CFRP's home visits. In most programs, 
home visits are designed to have a dual focus: (1) helping 
parents to become more effective in their role as educators , 
of their children; and (2) helping parents to meet a. broad 
range of family needs . and concerns. There appear to be 
differences, among the eleven programs in the relative empha- 
sis -t^hat is placed on parent education and family needs. The 
evidence suggests that home visiting staff in some programs 
devote only minimal attention to parent education or child 
development concerns. This is not true in all programs, how- 
ever. In fact, at two sites,' the dual focus of home visits 
is explicitly recognized, and separate family workers are ■ 
assigned the responsibility for each aspect. Two different 
models of infant-toddler home visit assignments are currently 
in place within local CFRPs': 

0 The Team Model — employed in Jackson and New 

Haven — was developed to ensure that both parent 
education concerns and family needs are addressed 
adequately in home visits. Visits are conducted 
by two family workers: one has responsibility 
for working with the parent and child on issues 
related to the child's development ' and parenting 
skills; the other focuses more broadly on family 
needs . - 

0 The Single Worker Model — employed at all other 
'sites— assigns one family worker to each family, 
with responsibility for both aspects of the 
home visits, child development;^ and parentirig 
issues as well as family needs. 

Home visit emphasis is determined to some extent 
by the type of curriculum that is used to guide home visit 
activities. Only four programs have adopted a developmental 
curriculum. In the other seven programs , home visits are 
planned by family workers themselves. This effort is 
closely supervised in three of these programs/ usually by 
someone with a background in child development; family 
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workers in the other four programs receive little or no 
guidance concerning the types of activities that visits 
should -cover . It is of interest to note that there appears ^ 
to be a relationship between the home visit planning effort 
and the frequency with which home visits occur; frequency . 
decreases when home visiting staff do their -own planning 
and have no curriculum or supervisor to fall back on (Table 

2-10). ^ 

(. 

The fact that greater emphasis appears to be placed 
on family* needs in the home-based activities of some programs 
may be related to the background of family workers . Their 
training tends to be in social work or related fields, rather 
than in parent education or child development. A substantial 
proportion (56%) of family workers at the ^six impact study 
sites perceive a need for additional training in these areas., 

Table^2-10 

Relationship between Home Visit 
Frequency and Curriculum 

Home Visit Frequency 

. y Once a More than 

month once a month 

No curriculum or specific heLp 

^ Oklahoma City , ' x- 

- Schuylkill Haven x 

Curriculum 

Jackson x ^ 



Las- Vegas 



:x 



Poughkeepsie x 
Specific help 



X 



Gering 
Modesto 



^PrograiTis with varied home visit frequency were excluded from 
thes^ analyses. 



m 

* Phase II Program Study Report , February 1980 . 



There are factors other than those related to staff 
and pro.gram characteristics that influence home visit focus. 
The types of activitie^--that take place are dictated largely 
by the needs and goals that in^vidual families identify or 
set for themselves. It is up to parents to decide what they 
want out of the program, a philosophy that is inherent in the 
CFRP concept. As impact study 'data indicate, the majority of 
families appear' to be more concerned with getting help with 
such practical needs as housing, employment, and health than 
with child development or parenting skills.* (This is the 
case at least in the early stages of families' involvement 
in CFRP; the types of goals that are set may change after 
families have been in the program for longer periods of time . ) 
In some programs parents also have a choice in deciding what 
program activities- they want to participate in and with what 
frequency, factors that undoubtedly contribute to low parti- 
cipation levels in center-based activities. Yet few programs 
have developed special home-based activities for families that 
do not attend center sessions" to ensure that appropriate parent 
education services are provided in the home. While individual- 
ization (interpreted as parent choice) of program services is 
an explicit mandate of CFRP, it is not clear that it maximizes 
attainment of the child development-related objectives of the 
program. - 

2.3 Total Program Participation 

Figure 2-1 summarizes data on program participation 
by evaluation families presented earlier in this chapter. 
Shown are total participation rates, (including center sessions 
and home visits) for both "center" families (those who came 
to the center at least once per quarter) and "non-center" 
families (those who participated less frequently in center- 
based activities.) Total participation is consistently lower 



* Phase II Research Report , Fe^bruary 1980. 



for non-center families. As is .evident froit, the figure, 
treatment is, considerably more intensive in Jackson, St. 
Petersburg, and Salem than in Las Vegas and Oklahoma City. 

The re.l,^tiyely llbw participation rates of "non- 
center" families, and of nearly all families at certain sites, 
raises serious questions, as to whether CFRP can be expected 
to have an overall effect on children's development. While 
effects might be expected for active participants, any such 
effects might be diluted by nonparticipant families — and 
thus overall comparisons between program families and those 
outside the program might fail conventional statistical tests. 
As we have seen, this concern is underscored by two additional 
facts: (1) The content of home visits is not in all cases 
focused on child development; and (2) the methods by which 
center-based parent education sessions are conducted do not 
involve the kind of practical instruction that is likely to 
be most useful . The next chapter demonstraites that these 
concerns are- justified," in that CFRP has negligible develop- 
mental effects, but also that there are promising signs that 
active participation may convey developmental benefits. 
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Chapter 2 Note 



s 2-2 and '2-3 



Participation rates reported in Tables 
were tomputed as follows: Monthly counts of frequencies of 
participation were averaged to the quarterly l^vel, and quarter 
ly data "were then averaged across the six quarters comprising 
the 18-month period to arrive at overall par;}/icipation rates. 
In the event that data were missing for a qtiarter, . rates were 
computed based only on quarters for which data were available. 
In. cases where data were missing for one month of a quarter, 
data from the remaining two months were used to compute a 
quarterly participation rate. However, if data were missing 
for more than one month, data were considered missing for 
that entire quarter. These computational procedures differ 
from those used in earlier reports. Partly because of these 
changes in procedure, but particularly because most tables 
in this chapter include only "center" families who partici- 
pated at least once per quarter, participation rates reported 
here are higher than those reported previously. It should 
also be noted that CFRP staff did not always distinguish be- • 
tween infant-toddler- and parent education sessions when re- . 
porting- participation rates, so that total center partic\pa- 
tion figures (Tables 2-2 and 2-3) are more meaningful than 
either parent education or infant-toddler rates reported 
separately. 
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Chapter 3 



PROGRAM IMPACT ON CHILD DEVELOPMENT 



I Scores of 187 CFRP and 184 control/comparison group 
i children on the Bayley Scales of • Infant Development 
1 were compared across and within the six impact study 
I sites, employing both ANCOVA and value-added approaches. 
I None of these analyses revealed any significant differ- 
jences between the two groups, with the exception of a 
I positive effect of CFRP on mental growth at one site, . 
I assessed by the value-added method. However, analyses 
i of relationships between CFRP participation variables 
land Bayley scores within the CFRP gro.up indicate that 
I participation in infant-toddler sessions may be a key 
I element in enhancing child development. Whether con- 
Isidered alone, in- interaction with home visit rate,U.pr 
1 in interaction with a measure of family strengt/^s, ; 
Irate of participation in infant-toddler sessions is 
[ positively associated with children's development.. 



-This chapter .examines CFRP's impact on the develop- 
ment of the infants and toddlers who are the focus of this 
longitudinal evaluation. Their development was measured by 
thd Bayley Scales of Infant Development. Section 3.1 de- 
scribes the sample of children -tested and their performance 
on individual itenis in the Bayley Scales. Section 3.2 de- 
scribes the analytic approach employed. Comparisons of per- 
formance between CFRP children and those in the control/ 
comparison group on the Bayley Scales are^ reported in Section 
3.3. Relationships between level of participation in CFRP 
and outcomes for children are discussed in Section 3.4.^ 
Finally, Section 3.5 presents a Igrief summary. of the results. 
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3.J 

4 



3.1 Sample and Test Description 

* . . 

The Bayley Scales were administered to infants and 
■t ■ ■ 

toc^^lers at the six impact study sites over a period of six 

months' (October 1979 to March 1980). Testing was staggered 
sQSthat youngefv children would reach 15 months of age before 
b^ing tested. The vast majority (95%) of children tested 
were between tl^e \ges of 15 and 22 months. Children were . 
tested in. their own homes, since a pilot study revealed that 
in-home testing cou\d be . feasible with only minor modifica- 
tions of the instrument (described below). On-'site researchei: 
who underwent a one-week Bayley training "session conducted 
*these assessments;, tfester performance in Bayley administration 
was monitored throughout thp data collection period.. Overall , 
the quality of the assessment data was judged to be good. 
(Details of the monitoring process and analyses of data quali 
~ar-e^ - reported in Append ix A . ) 

3.1.1 The Sample 

Characteristics of the child sample were examined 
in order to determine whether there were differences from 
site to site, or differences between CFRP and control/com- 
parison children within or across sites. The most important 
difference revealed by this examination had to do with child 
•age. Ages of children at time of assessment were distributed 
differently across the six impact study sites ^ (Table 3-1). 
Age differences also wejre evident, within and acro.ss sites, 
between children in the CFRP and control/corrtparison groups.^ 
Overall, 61 percent, of the children tested were 18 months of 
age or younger.; a higher propottion of the control/comparison 
children (66%) than of CFRP children (56%) were in this 
younger group. This difference in age distributions had 
major analytic conseqjuences , as discussed later. 

\ 

' In addition, differences were detected' on selected 
family background charac'teristics both across and within 



Total sample 

Jackson 
CFRP 

Comparison 

Las Vegas 
CFRP 

Comparison 

New Haven 
CFRP 

Comparison 

Oklahoma City 
CFRP 

Comparison 

St. Petersburg 
CFRP 

Comparison 

Salem 
CFRP 

Comparison 



Table 3-1 

Ages of Children at Time of Testing 

Greater 

15-16 17-18 19-20 21-22 23-24 than 24 
months months months months months months Total 



115(31%) 111(30%) 70(19%) 55(15%) 15(4%) 5(1%) 371(100%) 



6(20%) 
8(40%) 



13(37%) 
U(38%) 



5(19%) 
3(21%) 



9(32%) 
9(21%) 



10(29%) 
15(37%) 



9(26%) 
17(4^) 



15(50%) 
6(30%) 



9(26%) 
8(28%) 



3(12%) 
2(14%) 



7(25%) 
18(42%) 



8(24%) 
19(48%) 



11(32%) 
5(13%) 



5(17%) 
4(20%) 



6(17%) 
5(17%) 



10(38%) 
4(29%) 



5(18%) 
7(16%) 



7(21%) 
4(10%) 



6(18%) 



2(7%) 
2(10%) 



6(17%) 
4(14%) 



5(19%) 
4(29%) 



5(18%) 
6(14%) 



7(21%) 
2(5%) 



6(18%) 
6(16%) 



2(7%) 
0 



1(3%) 
1(3%) 



2(8%) 
1(7%) 



1(4%) 
2C4%) 



1(3%) 
0 



2(6%) 
2(5%) 



0 
0 



0 
0 



1(4%) 
0 



1(4%) 
1(2%) 



1(3%) 
0 



0 

1(3%) 



30 
20 



35 
29 



26 
14 



28 
43 



34 
40 



34 
38 



4x 



sites/ Several characteristics were examined: per capita 
income, sex of the focal child, employment and marital 
status of the mother, whether the mother had graduated from 
high school, and whether the focal child was the oldest. 
Site differences were evident for marital status and whether 
the tested child was the oldest in the family. Some differ- 
ences also were detected between the CFRP and cdntirol/compkr 
son groups at selected sites: \ 

• child *s sex — La^s Vegas and Salem j 

• , mother* s employment status — Jackson, 

New Haven, Oklahoma City, St. Petersburg, 
and Salem 

• marital status — New Haven 

• whether the child is the oldest— 

^ New Haven, St. Petersburg, and perhaps 

Jackson 

3.1.2 Overall Performance on Bayley Items 

The Bayley instrument consists of two scales, a 
mental development scale (MDS) and a physical development 
scale (PDS). Items are normed by age. From^ the MDS a total 
of 69 items, normed for ages twelve to over thirty months, 
were selected for .administration to the study's sample of 
infants and toddlers. Table 3-2 shows the percentage of 
children passing each item, by age. With a few exceptions, 
most MDS items were passed with greater frequency by older 
than by younger children. (All items were retained for 
further analysis, in order to preserve comparability with 
published norm data for the scale as a whole.) 

Testers also administered 16 items from the PDS 
that required no special equipment (i.e., steps or walking 
board) and were therefore feasible to use in the home. Per- 
formance varied substantially on only a few items (Table 3-3 
Ten items that had adequate variation in pass/fail rates 
(indicated with an asterisk in Table 3-3) were retained for 
future analysis . 



Table 3-2 
Percentage Passing MDS ^Items 



Item 



101. 
102. 
103. 
104. 

105. 
106. 
107. 
108. 
109. 
110. 

111. 
.112. 
113. 
114. 
115. 
116. 
117. 

116. 
119. 
120. 

121. 

122. 
123. 
124. 
125. 
126. 
127. 
128. 
12p. 
I 

13b. 

131. 



1 
1 
1 

1? 



3(2. 
3, 



136. 
137. 
138. 
139. 
140. 

141. 
142. 
143. 
144. 
145. 
146. 
147. 

148. 
149. 
150. 
151. 
152. 
153. 
154. 
155. 

156. 
157. 
158. 
159. 

160. 

161. 
162. 
163. 



Jabbers expressively 
Uncovers blue box 
Turns pages of booX 
Pats whistle doll, 
in imitation 
Dangles ring by string 
Imitates words 
Puts beads in box (6 of 8) 
Places 1 peg repeatedly 
Removes pellet from bottle 
Blue board: places 1 round 
block 

Builds tower of 2 cubes 

Spontaneous scribble • 

Says two words 

Puts 9 cubes in cup 

Closes round box 

Uses gesture to make wants kno\ 

Shows shoes or other 

clothing or own toy 
Pegs place^d in 70 seconds 
Builds tower in 3 cubes 
Pink board: places round 

block 

Blue board: places 2 round 

blocks 
Attains toy with stick 
Pegs placed in 42 seconds 
Names 1 object 
Imitates crayon stroke 
Follows directions, doll 
Uses words to make wants known 
Points to parts of doll , 
Blue board : places 2 round 

and 2 square blocks 
Names 1 picture 
Finds 2 objects 
Points to 3 pictures 
Broken doll: mends marginally 
Pegs placed in 30 seconds 
Differentiates scribble from 

stroke 
Sentence of 2 words 
Pink board: * completes 
Names Q. objects 
Points to 5 pictures 
Broken doll: mends 

approximately 
Names 3 pictures 
Blue board: places 6 blocks 
Builds Tower ^ of 6 cubes 
Discriminates 2 objects 
Names 4 1/2 pictures 
Names 3 objects 
Institutes vertical and 

horizontal strokes 
Points to 7 pictures 
Names 5 pictures 
Names 2nd picture 
Pink board: reversed 
Discriminates 3 objects 
Broken doll: mends exactly 
Train of cubes 
Blue board: completes in 

150 seconds 
Pegs ^placed in 22 seconds 
Folds paper 

Understands 2 prepositions 
Blue board: completes in 90 
seconds 

Blue board: completes in 60 

seconds 
Builds tower of 8 cubes 
Concept of one 
Understands 3 prepositions 



Normed 


15-16 


17-18 




A.^ — * ** 


rvx X 


age 


months months 
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Table 3-3' 



Percentage Passing PDS/ Items 



Item 



Normed 
age 



47. 




Stands up: I 


12.6 


48. 




Throws ball ^ 
Walks sideways / 


13.3 


49. 




14.1 


50. 




Walks backward, few steps 


14.6 


51 . 




Stands on riaht foot / 
with he Id / 


15.9, 


52 . 


* 


Stands on left foot t 
with help / 


16 . 1 


57 . 


* 


Stands ud: II / 


21.9 


58. 


* 


stands on left foo^ 


22.7 


59. 


* 


Jumps off floor, 
both feet ■ 


23.4 


60. 


* 


Stands on right 
foot alone 


23.5 


61. 


* 


Walks on line, 
general description 


23.9 


65. 


* 


Walks on tiptoe, 
few steps 


25.7 


68. 


* 


Walks backward. 10 feet 


27.8 


71. 




Stands up: III 


30+ 


73. 




Walks on tiptoe, 10 feet 


30+ 


75. 




Walks on line 10 feet 


30+ 



15-16 
month's 



61 

61 
21 
28 

5 

23 

35 

8 
4 
4 
0 
1 



17-18 
months 




69 

71 
38 
38 

13 

29 

36 

8 
22 
4 
1 
0 



19-21 
months 



22-24 
months 



All 

children 



100 


N=40 


N=366 


92 


95 


94 


97 


9o 




82 


82 


82 


88 


93 


• Q"7 


75 


93 


/I 


77 


90 


71 


51 


51 


38 


47 


53 


39 


24 


55 


18 


41 


53 


33 


53 


73 


45 


21 . 


35 


. 15 


18 


24 


16 


H 


17 • 


7 


4 


0 


1 


0 


0 


0 
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^Items , combined into a Physical Score that were used in analyses. 
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3.1.3 The Test^ Environment 



At the conclusion of each Bayley assessment, a 
Child Behavior Record was completed. This record describes 
the test environment, as well as the social behavior- of the 
child (and parent, if present) . The results are summarized 
in Table 3-4. As shown, most children engaged willingly in 
the test, and were friendly toward the examiner, with some 
initial wariness. A few behaviors differed by age, site, 
and group: 



• Easy acceptance of the test was more 
frequent in older children (62%); 
frequent verbalization was observed 
more often in older children (30%) 
than in younger children (11%). 

• Far more parents in Jackson, Las Vegas, 
and New Haven assisted children fre- 
quently (27%) than in Oklahoma City, 
St. Petersburg, or Salem (4%). 

• More CFRP children were judged as 
easily distracted (23%) than control/ 
comparison group children (14%); and 
18% of CFRP parents frequently assisted 
or interfered in the test administra- 
tion compared to 9% of control/com- 
parisori group parents. 

Problems noted in the test environment were the 
presence/interference of other children (15€); interruptions 
due to the flow of people in the home (11%); noise due to 
television, stereo, or people (11%); heating or lighting 
problems (4%); and lack of availability of adequate testing 
surfaces (11%). However, 78 percent of the home test environ- 
ments were free of any kind of problems, and the observed prob 
lems were not found more frequently for any age group or 
site, or. in CFRP or control/comparison group families. 
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Category 



Table 3-4 
Infant Behavior Record Summary 

Review of Responses 



1. Social orientation 
(responsiveness to examiner) 

2. Cooperativeness with examiner 



Tearfulness 



6 



Goal directedness 
(persistence) 

Communication skills 



Parent behavior — test 
assistance 



Parent behavior — emotional 
support 



8. Judgment of test 



17% hesitant; 53% accepting; 28% frieindly; 
2% inviting 

8% refused many items; 36% refused orj re- 
sisted on one or two items; 25% accepted 
tests willingly; 30% enjoyed and readily 
performed test items 

37% displayed no. apparent fear; 40% some 
restraint during early portion of test; 
14% moder.ate restraint during first h^lf 
of test; 8% moderate restraint during 
much of the test 

19% easily distracted; 54% fairly per-j 
sistent; 27% very persistent | 

5% silent throughout testing; 50% | 
occasional vocalization; 26% frequent | 
vocalization/few words; 18% frequent 
verbalization 

^ t 

45% no assistance; 41% occasional assis- 
tance or interference; ^14% frequent ■ 
assistance or interference 

28% frequent positive support; 60% 
occasional positive support; 5% no 
comments to child; 25% occasional 
negative comments; 3% frequent negative 
comments 

8% fairly adequate; 36% average; 
54% very good 



, O-More than one category checked for some children. 
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3.2 Analytic Approach 



To' get a preliminary >picture of the performance of 
CFRP and control /comparison children against a ijackground of ' 
national norms, we computed normed scores for the mental . 
development scale. (Such scdres are computed so that the 
average score for children at every age will be 100; scores 
over 100 indicate that a child is developing more rapidly 
than average—performs above his or her age level— while 
scores below 100 indicate performance below age.lev^.) 
Comparisons of physical scores to national norms coul^d not 
be made because the physical development scale was no\t 
administered in its entirety. 

Normed scores were not used in analyzing the^ effects 
of CFRP, primarily b^.cause of complications introduced! by 
the age differences between CFRP and control /compari sob 
groups, noted earlier. (This issue is discussed further in 
Section 3.3.) Ra'ther, a Mental Score consisting simplj^ of 
the raw number of mental items passed and a Physical Score 
consisting of the proportion of physical items passed' Wed 
the basis for all further work. (A proportion, rather 1 than 
a sum, was used in the case, of the physical items, becAuse 
data on one or two items were missing . for some childreri: 
365 Physical Scores, 348 are based on all ten items, iS 
nine items, and 2 on eight. Proportions largely elimin^ate 
any spurious differences in scores that would arise becj^use 
o^ variations, in numbers of items . ) , 

Mental and Physical Scores were further scrutinized 
in several ways: First, their reliabilities were assessed. 
("Reliability" denotes the degree to which the different 
items" on a test give consistent results, and the degreejto 
which the test gives consistent results when re-administjered- 
that is, the degree to which it is free of measurement 



of 
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error.*) Reliabilities of both Mental and Physical scores 
were judgedXto be-high.** Second, the variability of the 
Mental, and Physical Scores was examined and found to be . 
sufficient to suggest that further analysis to -determine its 
sources would be fruitful. Third, the data were screened 
for "outliers" — children who were significantly older than 
the rest of the sample, at the time of assessment, or those 
whose scores wet^^ suspiciously high or low in light of their 
age. (Such cases were .omitted from further analysis.) 
Finally, -relationships (correlations) between Menta^-and 
Physical Scores (Table 3-5) and between children's ages at 
the time of assessment and their Mental and Physical Scores 
(Table 3-6) were ex^ined, within sites and within CFRP and 
control/comparison groups, in an effort to detect anomalous 
relations;tips that might warrant special analytic treatment. 
Most of the observed correlations were high, as expected; 
however, the . results also pointed to one anomalous subsample 
(in St. Petersburg), which was deleted from some' later analy- 
ses. ^' 

- In order to compare the Physical and Mental Scores 
of CFRP and control/comparison children, it was necessary to 
adjust or control for differences between the two groups in 
age at testing, as well as differences in 'family background 
characteristics, noted earlier. Two different techniques 
were used to make the necessary ad justments . One, called . 
analysis of covariance, or ANCOVA, is rather like a handicap 

*Reliability is estima^ted by an internal consistency measure, 
alpha. Alpha varies from 0.00 to 1.00. An alpha of 0.00 ind 
cates a completely unreliable measure; an alpha of 1.00 in- 
dicates a measurd that is perfectly reliable. Thus, an alpha 
of .95 indicates that 95 percent of the measured variation 
among the scores of children is attributable to genuine diffe 
ences, and that only 5 percent of the measured variation is 
due to random effects or measurement error. 
**Alphas for Mental Scores ranged from a low of .88 in Oklahoma 
City and St. Petersburg to a high of .95 in Las Vegas. 
Alphas were somewhat lower for Physical Score, ranging from 
a low of .72 in New Haven to a high of .85 in Las Vegas. 
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- Table. 3-5 
Mental and Physical Score Correlations 









Cot 


relation 




CFRP 


Comp . 


CFR 


^ Gomp . 


Jackson 


26 


16 


..06 


\ .55 


Las Vegas 


29 


21 


.59 


\ .74 


New Haven 


21 


10 


.01 


\87 


Oklahoma City 


23 


37 


.63 


.3^ 


St. Petersburg 


29 


36 


.64 


.. 14\ 


Salem, 


29 


35 


.34 


.59 \ 



/ Table 3-6 

Corirelations of Child Age With Mental 
and Physical Scores 

/ 

I 

Mental Score Physical Score 







N 


Correlation 




N 


Correlation 




CFRP 


Comp . 


CFRP 


Comp . 


CFRP 


Comp . 


CFRP 


Comp . 


Jackson 


30 


. 16 


.43 


.70 


26 


16 


-.06 


.79 


Las Vegas 


32 


25 


.75 


.73 


29 


21 


.64 


.60 


New Haven 


21 


9 


.15 


.66 


21 


10 


.49 


.65 


Oklahoma City 


25 


37 


.80 


.72 


23 


37 


.66 


.38 


St, Petersburg 


29 


31 


.70 


-.01* 


29 


36 


.52 


.17 


Salem ■ 


27 


33 


.82 


.83 


29 


35 


.18 


.62 



*The correlation between Mental Score and age within the St. 
Petersburg control/comparison group is somewhat anomalous. 
Ordinarily, high correlations between age and Mental Scores 
are expected/ although these may be lessened when children 
receive special treatment — as in the case of the Jackson 
CFRP group. Closer examination showed that the St. Peters- 
burg correlation 'was not an error but was due to a group of 
young children who received apparently genuine, very high 
scores. Because of this anomaly, St. Petersburg was excluded 
from many comparisons between CFRP and control. New Haven 
was also excluded because the correlation within the CFRP 
group seemed too low to be explained by a treatment effect. 



in golf or bowling: before individuals' scores are compared, 
an adjustment is made to compensate for known' or suspected 
inequities in initial skill. The second, called the value- 
added technique, uses data from the comparison group to 
predict how CFRP children with given characteristics would 
develop in the absence; of "treatment . " The difference 
between their actual and predicted performance is the "value 
added" by CFRP. These analytic techniques ai^ discussed in 
more detail in Appendix B. They are mentioned here to 
stress the point illustrated concretely later, that results 
are not always identical when different analytic approaches 
are used. 

All analyses were conducted both within sites — to 
examine differences due to program implementation, client - 
populations, and other site characteristics — and across 
sites — to maximize 'statistical power and assess the impact 
of the program as a whole. 

3.3 CFRP's Impact oh Child Development 

I,n^ assessing CFRP ' s impact on the development of 
infants and toddlers, we first compared the Mental Scores o£ 
CFRP and control /comparison group children against national 
norms.. As noted earlier , similar comparisons could not be 
made for Physical Scores because the physical development 
"scale was not administered in its entirety. The CFRP and 
control/comparison group children combined scored slightly 
below national norms (97.2 compared vrith 100) in terms of 
their mental development; the resultfs ^ were quite similar 
across sites (Table 3-7). Performance with respect to norms 
was less satisfactory for older children at all sites. This 
finding parallels results from afnumber of other studies of 
infant intervention programs, including the Parent-Child 
Development Center (PCDC) program. 
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Table 3-7' 
Normed Mental' Score PeVtormance^ 



Age in Months 





N 


15-16 


17-18 


19-21 


22-24 


^ Total 


Jackson 
CFRP ■ 
Compalrison 


30 
16 


102.3 
105.2 


100.8 
98.8 


84 . 2 
94.0 


71 . 3 
n . a . 


94. 8 
•. 100.8 


Las Vegas 
CFRP 

Comparison 


33 
25 


1.04 . 0 
104.0 


94.6 
97.7 


91.6 
95.3 


87.0' 
90 . 0 


96.8 
98. 5 


New Haven 
CFRP 

Comparison 


22 

10 


118.4 
115.0 


98.0 
114.0 


87.8 
.94.0 


73.0 
85 .7 


93.2 
101 .8 



Oklahoma City 

CFRP 25 
Comparison 38 



103 , 
113 



92 
-97 



7 

0 



84.0 
89.6 



79.6 
85,8 



,91 .3 
97.0. 



St. Petersburg 
CF^P 

Comparison 



32 



103 .3 
115.6 



81 .3 
99.8 



96.8 
90.2 



85 . 3 
n. a . 



96.0 
104.4 



Salem 

CFRP 28 110.2 105.2 101.7 91.7 104.3 

Comparison 34 106.8 100.0 94.2 87.6 99.9 



ERIC 



37 



The data presented in Table 3-7 s^ag^est that the 
scores of control/comparison children in most age groups and 
at most sites are higher than those of children in the CFRP 
group. Thi's conclusion is erroneous because of the differ- 
ential distribution of child ages both across and within 
sites. Children in the control/comparison group were . . ^ 
generally younger than those in the CFRP group, and noi^med 
scores d'ecrease as children get older. Also note that 
apparent grpup differences are virtually nonexistent when 
average Mental Scores are computed for> each group by site 
(Table 3-8). Similar descriptive information concerning 
Physical, Scores is presented in Table 3-9. .... 

Mental and Physical Scores adjusted for child age 
and other important family background characteristics were 
used to assess CFRP\s impact. Across-site analyses using 
both the ANCOVA and v.§Llue-added analytic 'techniques provide 
no evidence that CFRP hais had a positive impact on either the 
mental or physical development of children. The results are 
summarized in Tables 3-lb and 3-11. Three sets of data are 
presented in the tables. \ The first two' columns are results 
from ANCOVA analyses. Th^ net score change/ simply reports 
how many points higher (or lower) — on average — CFRP children 
scored on Mental Score compared with children in the control/ 
comparison group. On Physical Score one-tenth of a point 
represents one test item (this is because proportions rather 
than number of items were used to compute this score)* The 
change iri' growth rate indicates the degree to which children' 
scores grow more rapidly than normal .as a result of participa 
tion in CFRP. The third column reports results of the value- 
added analytic approach; results are similar to net score 
changes. None of the ^results reported in either table were 
statistically or substantively significant. (Details con- 
cerning these analyses are presented in Appendix C ) 



Table 3-8 



Mental Score Descriptives 



N 



Mean 



S.D. 



Jjackson 
Las Vegas 
New Haven 
Oklahoma City 
St. Petersburg 
Salem 




CFRP 


Comp . 


CFRP 


Cortip . 


GFRP 


Comp . 




16 


121 . 9 


122. 0 


5.61 


7.60 


32 


25 


121.7 


123.5 


7.35 


8.70 


21 


9 


124.8 


125.1 


5.78 


5.90 


25 


37 


122.1 


123 .0-^ 


5.95 


. 5.40 


29 


31 


123.6 


121.8. 


7.70 


2.65 


27 


33 . 


126.3 


123.6 


7.84 


2.11 




Table 
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Physical Score Descriptives 



N 



Mean 



Jackson 
Las Vegas 
New Haven 
[klahoma City 

Petersburg 
SaLem 



S.D. 



CFRP 


Coinp. 


CFRP 


Comp . 


CFRP 


Comp 


26 


16 


.32 


.39 


.16 


.26 


29 


21 


.41 


.45 . 


.21 


.26 


21 


10 


.60 


.60 


.22 


.26 


23 


37 


.57 


.58 


.22 


.22 


29 


36 


."49 


.44 


. .18 


.21 


29 


35 


.60 


.58 


.21 


.20 



Table 3-10 

CFRP Effects on' Mental Score Across Site&,, 



Effect 

S.E. 

£^value 



ANCOVA 



Net Score 
Change 
-0.29 
0.38 
>.50 



'Change in 
Growth Rate 

0.01 

0.16 
,49 



Value Added 
-0.25 
0.69 
>.50 



Table 3-11 \ 
CFRP Effects on Physical Score Across Sites 
ANCOVA 

iThange in 

Value Added 



Effect 

S.E. 

£-value 



Net Score 
Change 



Change m 
Growth Rate 



-(.002 
.014 
>.50 



-.009 
.006 
>.50 



-.019 
.019 
>.50 



Finally, wi thin-site analyses were performed on 

Mental and Physical Scores. Only Mental Score analyses using 

the predictive (value-added) approach in Salem suggest that 

CPRP has had an impact (Table 3-12), with a net gain of 

about two points on Mental Score. At ,eighteen months, a net 

average gain of two points would put the CFRP children at or 

above the Bayley Mental Score norm, rather than just below 

it. No CFRP impact on children's physical development was 

detected at any of the sites. 

Table 3-l"2^ 

Average CFRP Effects on Mental Score, 
Within-Site Value-Added Analyses 

Jackson Las Vegas Oklahoma City Salem 

Effect -1.78 -0.71 -1.86 2.18 

S.E. 0.94 0.87' 0.63 0.87 

df^ 28 28 22 26 

£-value >.50 >.50 >.50 .04 



^New fiaven^ and St. Petersburg excluded: see note attached to 
Table 3-6 for an explanation. 

^Significance tests controlled for fdur simultaneous one- 
tailed tests . 

3.4 CFRP Participation and Child Development 

Although there are no significant differences, on 
average, between the Bayley developmental scores of CFRP and 
control/comparison group children, we cannot conclude that 
CFRP has no effect on children's development. CFRP is not a 
uniform, standardized program; services' are tailored to the 
needs and preferences of families (that is, they are "individ- 
ualized"), and there is considerable variation among sites 
in program structure and activities.* Thus different 



See ghase III Program Study Report , August 1980. 



families and. children "participate" in CFRP in different 
ways. Moreover, as shown in Chapter 2, families differ 
widely in their frequencies of participation in program 
activities. It is quite possible that these differences 
in type and amount of pa,rticipation are related to the 
developmental benefits children derive from the program. 
This overall comparison may be concealing program effects 
that occur only when the child participates in certain ways, 
or participates often enough or long enough. 

Numerous analyses were performed in order to ex- 
plore these possibilities. Details are reported in Appendix 
D; major findings are summarized here. On balance, results 
suggest that developmental benefits, as measured by the Bayley 
mental and physical scales, may in fact relate to the amount , 
and type of the family's participation in the program, al- 
though this conclusion is clouded by differences in partici- 
pation rates from site to site. 

3.4.1 Effects of Amount and Rate of Participation 

The simplest view of the relationship between par- 
ticipation and developmental benefits is that "more is betters- 
more participation will lead to greater benefits. (There are 
reasons why this view might not hold for CFRP; we shall 
examine these reasons ;tater.) There are three types of CFRP 
activities to which the maxim "more is better" might apply: 
parent education center sessions, infant- toddler center 
sessions, and home visits. As shown in Chapter 2, all fami- 
lies receive home visits, but with varying frequency. Many 
do not attend cen\er sessions at all, and, among those who 
do, there is substantial variation in the frequency of their 
attendance. Consequently, effects of monthly rates of par- 
ticipation in par^t education sessions, infant-toddler 
sessions, and home visits were examined separately. 
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Summary statistics for these three measures are presented in 
able 3-13.* 



The participation measures were examined in rela- 
tionship to five different me-a-sures of growth: Bayley scale 
mental . growth indexes calculated (for the entire sample) by 
the value-added and ANCOVA techniques, physical growth in- 
dexes calculated for the entire sample by both techniques, 
and a mental growth index calculated by the value-added 
technique for the Salem sample. ■ (Recall that valu^-added 
scores in Salem were the only mental growth indexes at any 
site to show a positive overall effect of CFRP enrollment.) 



Table 3-13 

Summary Statistics, Participation Measures^ 

(per month) 





Minimum 


Maximum 


Median 


Mean 


S.D. N 


Home Visit 












Rate 


0.0 


3.9 


1.1 


1.4 


0.8 143 


Infant-Toddler 












Session Rate 


0.1 


2.2 


1.0 


^1.1 


0.5 61 


Parent Education 












Session Rate 


0.2 


1.7 


1.0 


0.8 


0.-4 66 



^For each family, participation measures cover the period 
preceding testing of the child. \ 



*The Bayley scales had to be administered Across a period of 
six months in order to ensure that all chi^ldren would be at 
least 15 months old when tested. Consequently, there were 
appreciable differences in amounts of time that different 
families had been in CFRP prior to testing. Participation 
data used m all analyses in this chapter reflect these 
differences; participation rates for each family are based 
only on the period preceding testing. In additicsn, the 
analytic sample included all available families, riot just 
those whose participation rates for various activities 
exceeded once per quarter. Consequently monthly participa- 
tion rates shown in Tab^le 3-13 are not directly translatable 
into the quarterly rates presented in Chapter 2, which' cover 
participation over the same time period, regardless of ^hen 
the child was tested. \ 
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The two physical measures were so highly correlated (.87) as 
to be almost indistinguishable. The two types of mental 
measure were also highly correlated (.68),, but were suffi- 
ciently distinct that they might prove sensitive to different 
participation measures. 

Relationships between the three participation in- 
dexes and the five developmental indexes are shown in Table 
3-14. There are enough significant and near-significant 
correlations in the table to lend credence to the general 
view that program participation affects both mental ,and 
physical development of children. Specifically, although 
the pattern of findings is quite complex, it suggests that 
attendance at ' infant-toddler center sessions ig positively 
associated with mental growth as measured by the ANCOVA 
method and with physical growth measured by both methods. 

However, caution must be used in interpreting the 
association between participation and the developmental 
measures. Recall from Chapter 2 that there are substantial . 
differences in participation from site to site. Because of 
these differences it is possible that what appears to be a 
relationship between- participation and . developmental benefits 
is instead a relationship between site and developmental bene- 
fits. The only definitive way to disentangle these rival 
interpretations would be to determine whether there is a re- 
lationship between developmental benefits and participation 
within sites. Unfortunately, sample sizes within sites do 
not offer enough statistical power to perform such analyses 
with any confidence. Thus both interpretations remain equally 
consistent with the data; this fact must be kept in mind in 
reading the results of more detailed analyses of participation 
for the sample as a whole, which are reported below. 



Table 3-14 



Correlations Between Participation Measures 
and Mental; and Physical Growth Indexes 



Home Visit 
Rate 



Infant- 
Toddler 
Session 
Rate 



Parent 
Education 
Session 
Rate 



Center 
Partici- 
pation/Non- 
participation 



Mental Score 
Value-Ac'*ded 
N 

£-value 

ANCOVA Mental 
Score Residual 
N 

£-value 

Within-Salem 
Mental Score 
Value-Added 

£-value 

Physical Score 
Value-Added 

£-value 

ANCOVA Physical 
Score Residual 
' N ^ 
£-value 



-.02 

91 
> .50 



.06 
97 
. 25 



27 
45 



.11 

105 
> .10 



.17 
98 
.044 



41 
>.25 



39 
40 

.006 



.19 

18 
> . '23 



32 
46 

015 



27 

39 
049 



-.05 



> . 50 



05 
44 

,35 



20 
12 

,25 



-.07 
47 
> . 50 



-.16 
47 
>.50 



.06 

94 
>.25 



-.00 
100 
> .50 



.25 
27 
> .10 



.20 
108 
.017 



.13 

101 
.09 



^Significance levels are given for individual, one-tailed 
tests. As is well known, individual significance tests can 
be misleading when many teats are performed at once. (Five 
out of 100 will achieve conventional "significance" levels 
by sheer chance.) Across five simultaneous tests, a uni- 
variate significance level of .02 is actually signj.ficant 
at .10. Across twenty-five simultaneous tests, a univariate 
significance of about .004 is required for overall signifi- 
cance at .10. 
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3.4.2 



Interactions Among Types of Participation 



Closer examination of the simple hypothesis that 
"more (participation) is better" suggests that it may be too 
simple for CFRP. The various program elements are intended 
to work in concert with one another, not in isolation. Thus, 
for example, it might be the case that the combined impact of 
frequent home visits and frequent center attendance is far 
greater than the sum of the two effects taken separately. 
Statistically, this synergistic combination of effects is an 
"interaction. " 

One way to test for the presence of interactions 
irs^to form the product of the two participation measures in 
question^and examine the correlations of these products with 
the various growth measures . Table 3-15 shows relationships - 
between the growth indexes and the combinations (interactions) 
of home visit rates with the two center participation indexes. 
Thus, for example, the upper left-hand cell of the table shows 
the combined effect of home visit rates and inf ant^-toddler 
session rates on the ^Bayley mental growth index, calculated 
by the value-added technique. Several correlations are 
significant or near-significant, despite the fact that the 
need to combine variables for this analysis resulted in a 
reduction of sample size, hence of significance level, for 
each correlation. Thus, the table points to the possible 
importance of the combined effects of home visits and 
infant-toddler sessions, as well as the combined effects of 
home visits with some center participation on the part of 
parents. Subject to the qualification about site differences 
raised above, participation in center sessions, particularly 
infant-toddler sessions, seems to promote mental and physical 
development— and home visits appear to interact with center 
participation to give an added "boost" to Bayley scores. 



45/ 

00/ 
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Table 3-15 

Correlations Between the Combined Effects of 
Home and Center Participation and Mental and 
Physical Growth Indexes 

Combined Effect (Interaction) of Home Visit Rate and 



Infant- 
Toddler 
Session 
Rate 



Parent 
Education 
Session 
Rate 



Center 
Partici- 
pation/Non- 
participation 



Mental Score 
Value-Added 
N 

value 

ANCOVA Mental 
Score Residual 
N 

£-value 

Within-Salem 
Mental Score 
Value-Added 

p-value 

Physical Score 
Value-Added 
N 

£-value 

ANCOVA Physical 
Score Residual 
N 

, a 
£-value. 



-.02 
40 
>.50 



,41 
39 
.005 



.18 
18 
> . 20 



,13 
45 
,15 



,24 
38 
,074 



-.05 
41 
> .50 



05 
42 
35 



10 
12 
35 



17 
45 

,50 



.01 
45 
> .45 



.03 
91 
> . 35 



.00 

97, 
.50 



25 
27 
104 



.12 

105 
.106 



.15 
98 
.076 



^Sign4.f icance levels are given for individual, 
one-tailed tests. 
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3.4.3 Effects of Family Needs and Strengths 

Still further consideration of the nature of CFRP 
suggests that the meaning of "participation" is conditioned 
on the needs and strengths of families. Because families 
participat^e voluntarily, they are likely to seek out the pro- 
.gram in times of particular need, and they are likely to ask 
for help in areas where their needs are greatest. Conversely, 
they are least likely to seek help in areas of their greatest 
strength. Thus, a casual (and misleading) analysis might show 
a negative relationship between program participation and the 
family's level of functioning. Clearly, to assess the effects 
of participation, it is necessary to consider families' needs 
and strengths as well. 

The amount of benefit that a child experiences on 
some outcome measure may also depend on the needs and/or 
strengths of the child's family. It might be the case that , 
children from the neediest families profit the most from 
CFRP. on the other hand, a certain amount 6f family support 
(strength) may be necessary for a child to benefit from the 
program. Both hypotheses are plausible; they imply opposite 
relationships between family needs/strengths and the degree 
to which CFRP benefits children, but they also imply that 
family needs and strengths cannot be ignored in assessing 
CFRP's impact on the child. 

To begin to approach the difficult problem of 
measuring the needs and strengths of families, we construct- 
ed a set of nine measures, based on parent interview data 
and staff reports. The nine measures are listed and defined 
in Table 3-16; basic descriptive statistics are also shown. 
The first two measures— HASSLED and NEEDS— are fairly straight 
forward, though global, measures of need. The next three 
measures— CFRP SUPPORT, RESOURCES and ENTHUSIASM— are equally 
straightforward measures of strengths. CD EMPHASIS is a less 



Table 3-16 



Needs and Strengths Measures 



Name and Description 

\ 

HASSLED — the extent to which one feels "hassled" generally, by people in various 
roles (doctor, neighbors, family, friends, and relatives). From fall 1978 parent 
interviews. (A higher score is more hassled.) 

NEEDS — an index of extent of family needs across many genet^al areas. From fall 
1978 and spring- 1979 staff reports. (A higher score indicates more needs.) 



Summary Statistics 



min =0.00 
mean = 0.52 
SD .= 0.34 



min 
mean 
SD 



0.39 
1.55 
0.50 



max 
median 
N 

max 
median 
N 



1.00 
0.50 
143^ 

2.86 
1 .60 
126 



00 



CFRP SUPPORT — the degree of support for CFRP involvement from family and friends. 
From fall 1978 and spring 1979 parent interviews. (A higher score indicates 
greater support.) 

RESOURCES — an assessment of the personal or social resources available to the parents, 
such as frequency of social contacts outside of CFRP, ties with extended family, 
and awareness of social services available in the community. From spring 1979 staff 
reports. (A higher score indicates greater resources.), 

ENTHUSIASM — the extent of interest in or enthusiasm for CFRP activities, across a 
number of specific items. From spring 1979 staff reports. (A higher score shows 
greater enthusiasm. ) 

SPOUSE HELPS — the extent of help from one's spouse (or live-in partner) in routine 
child care tasks. From fall 1978 parent interviews. (A higher score means more 



min 
mean 
SD 

min 
mean 
SD 



min 
mean 
SD 

min 
mean 
SD 



RELATIVE HELPS--li)ce SPOUSE HELPS, but for other relatives, 
interviews. (A higher score means more help.) 



From fall 1978 parent 



min = 
mean = 
SD = 



1.11 
2.31 
0.74 

0.25 
0.93 
0.33 



1.01 
2.29 
0.68 

0.13 
0.50 
0.24 

0.13 
0.58 
0.29 



max 
median 

max 
median 
N 



max 
median 
N 

max 
median 
N 

max 
median 
N 



3 .46 
2.32 
126 

1.67 
1.00 
126 



3.53 
2.30 
130 

1.00 
0.50 
68' 

1 .00 
0.62 

89^ 



OTHERS HELP — like the two measures above, but for other people helping; the natural 
logarithm of this variable was used. From fall 1978 >arent interviews. (A higher 
score means more help.) 

CD (CHILD DEVELOPMENT) EMPHASIS — a three-valued assessment of the extent to which 
child development was emphasized during the first six to eight months of CFRP. From 
spring 1979 staff reports. (Codes: 0 - not at all; 1 - somewhat; 2 - fairly strongly.) 



min = -2.08 max = -0.13 
mean = -1.58 median = -1.38. 



SD = 0.56 



68" 



0- 58 families 2-22 families 

1- 55 families N = 135 



^The relatively high incidence of missing data on the HELPS constructs seems to reflect either the unavailability of 
or \the failure to use others in helping with routine child care chores, or some combination. 
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straightforward measure of need; staff who placed special 
emphasis on^child development during the first months of the 
program appeared to be responding to perceived special needs 
on the part of children ^ ^The remaining measures — the HELP 
series — may reflect needs, strengths, or both. 

"We then used these nine measures as "covariates *• 
in a series of exploratory multiple regression analyses. 
Roughly, this statistical approach affords a means of 
holding needs and strengths constant in order to isolkte the 
effects of participation on mental and physical growtfi. On 
the whole, taking account of needs and strengths did jiot 
change the pattern of associations between participation and 
growth indexes that resulted* from the simple analyses 
reported earlier (Table 3-14). Regression does, however, 
add something to simpler- correlational analyses, in. that it 
allows us to estimate the magnitudes of the increments in 
growth resulting from different levels of participation: An 
increment of one infant-toddler session per month produced 
an increment of about 3 points in Bayley Mental Score, 
in the ANCpVA model (p=.06). Similarly, one additional 
infant-toddler session per mpnth produced an increment- of 
one item on the physical scale in either model (p=.05 for 
value-added; p=.10 for ANCOVA) . 

Additionally, more complex regression analyses ex- 
plored the effects of various combinations of participation 
variables, in tandem wit\ the measures of needs and strengths. 
Among the salient results \f these explorations were the 
following: (1) Greater RESOURCES (defined in Table 3-16) 
were "^associated with higher mental growth indexes. (2) When 
RESOURCES were held constant, the combination (interaction) 
of infant-toddler sessions and hojne visits was still a 
potent one: one additional home visit per month seems to add 
about 1 1/2 points on the Bayley mental scale to the increment 
in growth produced by infant-toddler session attendance 



alone, and an additional infant-toddler session per month 
seems to add about 1 l/2 points to the increment produced by 
home visits alone. (3) Children whose mothers had greater 
RESOURCES profited more from infant-toddler sessions than 
those whose mothers had few personal resources. 

3 . 5 CFRP and Child Development; Summary and Conclusions 

No average difference in mental and physical develop- 
ment, as measured by the Bayley scales, was found between CFRP 
and control/comparison children after the former had partici- 
pated in the program for 12 to 18 months. Excepting fhe sug- 
gestion of a possible effect on Mental Scores at one site, CFRP 
has not (or notTyetl "produced more rapid or sustained develop- 
ment than could be expected in the absence of CFRP enrollment. 
However, this result does not necessarily mean that CFRP will 
never show effects, nor does it mean that a program like CFRP 
cannot enhance development in very young children. 

In an effort to explore further the potential of 
CFRP to enhance the development of infants and toddlers, we 
investigated the ^ef fects of program participation on indexes 
of mental and physical growth. The logic of our investiga- 
tion was that the program had its best chance to work among 
families who participated actively--and we knew that many 
did not. The investigation bore fruit. Hints of effects* of 
participation were scattered across five outcome measures, 
but all had a common thread: Participation in infant-toddler 
sessions seems to be a key contributor to the impact of CFRP 
on Bayley scores. Whether alone, in interaction with home 
visits, or in interaction with the personal resources avail- 
able to CFRP mothers, the rate of participation in infant- 
toddle^^ sessions is positively related to both mental and 
physical gains. 
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It is important not to overstate these findings. 
We have surninarized here only those results that point to 
potential effects of CFRP participation. Other analyses, 
giving no hint of CFR^^ef f ects , have been ignored in this • 
summary. In addition, ^already pointed out, the „possible 
effects of participation ai^ confounded with site differences. 
Nonetheless,, the pattern of results is suggestive. 
\ . 

Setting aside these neceagary caveats, the results 
appear to have several implications for the operation of 
CFRP. The sobering and overriding conclusion is that, while 
there are^^ indications that the program has the potential to 
enhance the development of children under age two, CFRP is a 
long way from realizing that potential. Typical participation 
rates are too low for the program to have much effect. (In 
the* current evaluation sample, roughly two- thirds of the CFRP 
families attended infant/toddler sessions less than once per 
quarter during their first 18 months of enrollment. Among 
those who did participate, the median rate of attendance was 
once per month.) To increase the effectiveness of the program, 
ways must be found to increase attendance at infant-toddler 
sessions. The effectiveness of such participation can be 
"levered" by frequent and regular home visits. Finally, con- 
sonant with CFRP's philosophy of working through the family, 
increasing the personal and social resources available to the 
mother — e.g., increasing her awareness of community services, 
or helping her establish and maintain ties with neighbors and 
her extended family--can further "lever" the developmental 
effects of home visits and infant-toddler sessions. All of 
this is more easily said than done, but appears to be necessary 
if the program is to achieve its goals for children. 
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EXECUTIVE SUMMARY 

CFRP was designed "to assist parents to promote 
the total development of children." A summary of findings 
concerning the impact of CFRP ' s infant-toddler component on 
the development of children* between the ages of 15 and 24 
months is presente^d below. 

Question 1: Is CFRP effective in promoting the development 
of infants and toddlers? 

NO. When tested after a year to ^a year and a half of program 
participation, CFRP children were not found to differ signifi- 
cantly, froip control/comparison children on mental and physical 
development scores of the Bayley Scales of Infant Development 
(BSID). t^ere is an indication of positive CFRP. Impact on 
mental development at only one site. j 

One possible explanation is tha:t family participa- 
tion in infant-toddler activities, particularly center 
sessions, is "less than optimal." There also is some 
queistion as to whether home visits occur with sufficient 
frequency to carry out an effective infant- toddler program 
in the home. Frequency is dictated to a large extent by 
high family worker caseloads, exceeding 20 families at some 
sites. 

Question 2: Is family participation in infant-toddler 

program activities related in any way to child 
development? 

YES , apparently. Rate of participation in infant- toddler 
center sessions is positively associated with both mental 
and physical development scores of the BSID. \The evidence 
also suggests that the benefits children derive from these 
sessions may be re^lated to rate of participation iii home 
visits and to the availability of personal and social 
resources to the mother or family outside of CFRP. That 



is, the positive effect of these sessions tends to 'be 

lllllt\nTatT'.^''' "'^^ participation i^°hoL 

visits and greater availability of resources. 

It is important to point out that just about half 
of the CFRP families had never attended an infant-toddler 
center session as of the date of Bayley testing. Among 
those Who had attended one or more sessions, the median rate 
was one session per month (during the period in which 
sessions were attended). The finding reported above suggests 
that more active participation in program activities, could 
result in demonstrable differences between the CFRP and 
control/comparison groups on the mental and physical develop- 
ment of children, favoring the CFRP group. 

There are several other issues which should be 
considered in connection with CFRP ' s effect— or lack of 
effect— on child development scores as measured by the 
Bayley scales. Fi^si of all, there is some question as to 
the appropriateness of this instrument for measuring CFRP 
•impact. Further, it is possible that CFRP impact on child 
development is simply too indirect to be detected after 18 
months of program involvement. Parents may have to change 
first, before their children are likely to be affected. If 
positive findings (of CFRP impact) from a pilot study 
involving observations of parent-child interaction are 
replicated-that is, if CFRP participation is effecting 
genuine positive changes in parent-child interaction-there 
is reason to believe that the eventual result will be 
(measurably) enhanced development of CFRP children.. 

If CFRP is not significantly affecting child 
development, there are several possible (and plausible) 
e'xplahati6ns. Amont them are: typical CFRP approaches to 
parent education (relying on didactic methods more than on 
observation of modeled behavior and opportunities to practice 
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ne\vly acquired parenting techniques); the typical focus 
of home visits (on family needs rather than parenting and 
child development concerns, peirhaps partly associated with a 
lack of child development background on the part of family 
workers); the low frequency of home visits and the low 
levels of family participation in center sessions. Changes 
in educational approaches and shifts in home visit focus 
would presumably do much to strengthen CFRP as a chil^^ 
.development program; however, the second finding reported 
above suggests that increases in the frequency of contact 
between program and family might well do even more. 

\ 



/ 

/ 




/ 

I 

/ 



54 



Appendix A 
BAYLEY DATA QUALITY 



It is customary in large-^-scale data collection 
efforts, especially those involving assessments of child 
development; to gather information concerning the quality 
of data obtained by on-site research staff. Tester per- 
formance in test administration and scoring was monitored 
on an\ ongoing basis during the fall/winter 1979-80 Bayley 
data collection period. In this- appendix the monitoring 
process is described and the quality of the Bayley data 
reported. 

'The monitoring process started during tester 
training, when a trainer accompanied trainees on several 
practice visits, designed to evaluate trainee performance. 
Additional training and assistance was provided to trainees 
on the basis of these evaluations to ensure uniform test 
administration. This process was repeated on site as part 
of the data collection start-up process. No testers were 
permitted to proceed with data collection tasks until their 
performance was deemed to be satisfactory by monitoring 
staff. 

Once the data collection effort had gotten under 
way, on-site research staff were responsible for ongoing 
monitoring of data quality. Staff periodically accompanied 
each other on testing sessions. One person was responsible 
for both test administration and scoring of items, while the 
other person independently scored the child's responses and 
observed tester performance/ At the conclusion of the 
session, staff compared scores in the two test booklets and 
reviewed any problems that occurred during the course of the 
test administration. Scores were not changed, however, in 
this review process. Completed tests and monitor booklets 



were reviewed a second time by a data collection supervi^ 
in Cambridge. Discrepancies in scores were tabulated i 
order to assess data quality. Periodically, on-site d;^ta 
collection staff received feedback concerning their |ferfor- 
mance based on this review. // 

As is noted in Table A-1; a total of 8// Bayley 
testing sessions were monitored in this fashion; during the 
fall/winter 1979-80 data collection period, ar/ average of 22 
percent of all Bayley assessments. The number of monitoring 
visits ranged from a low of 10 (14%) in Sal^m to a high of 
20 (31%) in Las Vegas. 

Table A-1 
Monitored Tests 

Percent of 
N Total Tests 



Jackson, MI 




11 


Las Vegas , NV 




20 


New Haven, 




12 


Oklahoma Ci-^y, 


OK 


16 


St. Peter sblirg. 


FL 


12 


Salem, OR 




10 


Total 




81 



22 
31 
30 
23^ 
17 
14 

22 (site average) 



The Bayley consists of two scales — the mental 
development scale (MDS) and the physical development scale 
(PDS). The MDS contains a total of 69 items for children 
in the 15-month or older age range. The PDS is considerably 
shorter, with a maximum of 16 items (excluding those requir- 
ing the step stool and walking board).* The total number of 
items administered depends on the child's performance. An 
average of 32 MDS items were administered to children in the 
CFRP evaluation; for the PDS the mean nurt^ber of items was 13 

*See Chapter 1 for an explanation on why certain items 
were excluded from the PDS. 
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In order to assess Bayley data quality, we computed 
^^3Bpe mean number of scoring discrepancies that occurred on 
each Bayley . test, as recorded in monitor 'booklets . An 
average of 2.2 discrepancies were found on the two Bayley 
scales combined, an overall error rate of less than five 
percent. Four sites had fewer discrepancies per test, as 
noted in Tables A-2 (Las Vegas, Jackson, Oklahoma City, and 
Salem); the Bayley data obtained in New Haven and St.' 
Petersburg were of considerably lower quality, .with per-test 
discrepancies averaging 5.4 and 3.2 respectively. One of 
the New Haven testers was terminated on the basis of these 
, data for the January-through-March data collection period. 
Scoring discrepancies were more common on the MDS than on 
the other scale, due largely to differences in the number 
of items in each scale. 



Table A-2 ^ . 
Mean Number of Discrepancies per Test 





MDS 




PDS 


Total 


Mean # of Items 


32 




13 


45 


Jackson, MI 


.9 




.3 


1.2 


Las Vegas, NV 


.7 




.3 


1.0 


New Haven, CT 


4.5 




.9 


5.4 


Oklahoma City, OK 


.7 




.8 


1.5 


St. Petersburg, FL 


2.8 




.4 


3.2 


Salem, OR 


1.2 


/ 


.3 


1.5 


Site Average 


1.7 




.5 


2.2 



To examine the sources of scoring discrepancy 
more closely, we singled out portions of the test for 
which discrepancy rates were particularly high. Items or\ 
the Bayley are administered in the context of different 
situations. Each situation contains items that are 
similar. In the "Cube Behavior" situation (part of the 
MDS), for example, the child is first asked to unwrap a 



cube. If the item is passed, the child is asked to build a 
tower of 2, 3, 6, and 8 cubes. Other related items concern 
building a train of cubes and using a single cube to demon- 
strate understanding of the concept of one. Items are 
similarly grouped in situations on the PDS . Discrepancy 
rates varied substantially across situations. 

The MD^ contains 19 situations. Table A-3 iden-- 
tifies nine situations on which scoring reliability was 
relatively low. The figures reported in the last column 
were derived by adding the total number of discrepancies 
that occurred across sites on all items in each of the 
situations and dividing this total by the number of items 
contained in the situation. The figures thus represent 
average total discrepancies per item across the six sites. 

Table A-3 
Discrepancies by S 
MDS 

Situation and Item Descriptions i 

Jointed doll (follows directions 
and points to parts of body) 

Vocalizations and words* (jabbers, 
says words, uses words to make 
wants known) 

Misc. items (pushes car, dangles ring, 

uses gestures to makd'-iWants known,* 

attains toy with stick, and finds 
2 objects ) 

Verbal comprehension (imitates word,* 
shows shoes, other clothing .or toy) 

Mencjs broken doll 

Names objects (ball, watch, pencil, 
scissors, cup) 

Names and points to pictures 

Peg board (places pegs in board) 

Pink board (places shapes in puzzle )- 



ituations 



# of Items 
n Situation 



Average 
Total Dis- 
crepancies 
per Item 



5.0 



4.3 



3.2 



2 
3 



3.0 
3.0 



3 
6 
5 
3 



2.7 
2.5 
2.2 
2.0 
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The .situations reported in Table A-3 account for 81 percent 
of all discrepancies. On all other situations, the total 
number of discrepancies per item was less thaA 2. Three of 
the nine situations in Table A-3 (indicated by asterisks) 
involve judgments of the child's behavior on the part of 
the tester, rather than direct responses of the child to 
directions or questions. Unreliability of judgments of 
relatively unstructured behavior thus appears to be a 
contributing factor in the overall discrepancy rate. 

On the PDS, items combine into six situations. 
In five of the situations, the total discrepancy rate per 
item exceeded 2.0, as noted in Table A-4. 

Table A-4 
Discrepancies by Situations 
PDS 

/ Average 
/ Total Dis- 

/ # of Items crepancies 

Situation and Item Descriptions in Situation per Item 

Walking skill — pull toy 

(sideways, backward) 2 3.b 

Walks on line (forward, 

backward, on tiptoe) 5 2.6 

Balance (stands on one 

foot with help and alone) 4 2.5 . 

Stands up from floor alone 3 2.3 

Jumps from floor ^ 

Summary 

The overall discrepancy rate per test was low, 
even on situations with generally lower scoring reliability. 
Scoring discrepancies on the jointed doll situation (the 
single least reliable situation), for example, occurred on 
only one out of Qxght tests. The factors examined lead to 
the conclusion that the Bayley data were of high quality^/ 
Possible exceptions are found at two sites— New Haven afid 
St. Peter sbdrg-^which had a higher than average per-test 
discrepancy rate. 
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TESTS OB IMPACTS, MODELS, AND ANALYTIC STRATEGIES 

Any statistical test of a program's impact in- 
volves an implicit or assumed model of how that program 
influences the outcome being measured. Further, when the 
outcome is a developmental phenomenon (that is, some kind 
of growth or change which is expected to occur "naturally" 
and systematically over time), the test employed makes very 
strong assumptions about the growth pattern and/or the way 
in which a program affects growth. In many evaluation 
applications, implicit models of impact and/or growth. are 

not scrutinized ^carefully, resulting in tests based on 

questionable assumptions. At best, sbch tests are inappro- 
priate, and the significance levels reported are wrong. At 
worst, bias in the tests may provide misleading results : an 
effect may be inferred that in fact does not exist, or an 
important effect may not be detected ^ven when the research 
design appears to have sufficient powet* to detect an effect 
of practical significance. ' 

Section B.l of this appendix examines the models 
and assumptions implicit in a relatively common statistical 
testing procedure — analysis of covariance, or regression 
adjusted teWts . Another, more recently developed approach, 
"value-added^' modeling (see Bryk and Weisberg, 1976a and 
1976b, and Bryk, Strenio, and Weisberg, 1980), is examined 
in Section B.2. The rationale for choosing the tests 
reported elsewhere in the volume is laid out in. some detail 
in Section B-3. ^ Whirfe a number of subtle and complex 
problems are reviewed here, little mathematical detail is 
included; references to more thorough, more rigorous 
discussions are given. 
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Traditional Tests 



^To understand the model and assumptions, underly- 
ing traditional statistical tests of impact, consider a 
basic, one-way ANCOVA (lower case characters represent 
population parameters; upper case characters are either 
variables or numbers; subscripts perform their usual 
indexing function) : 

[1] ' Y. . = m + g . + b^X.^ + b^X. 2 + • • • + b^.^^ + e. . 

Simply stated, the score of individual i in group j (Y^j) 
consists of a grand mean (m) plus the group's deviation from 
the grand mean (g^; j = 1, J groups) plus a multiple 

regression contribution reflecting the effects of a set of 
covariates such as the individual ' s age, sex, etc. (the 
terms t^^^X^^; k = 1, K covariates), plus an "error** 

term (®j^j)* ^ typical null hypothesis is that the treat- 
ment group means (Y.) are equal. 

The formal assumptions of the test of this typical 
null hypothesis are usually stated in terms of the distribu- 
tion of the "error" and its relationship to other variables 
in the equation: e^^j is assumed to be distributed normally, 
independently of each other e^j with zero mean and unknown 
variance; that variance is assumed to be constant across 
all values of the other variables in the equation (note 
that this subsumes the assumption of equal "error" variance 
within treatment groups); e^j is independent of (hence, 
has zero covariance with) all the variables in the equation. 
When these conditions are met, the statistical test of the 
stated null hypothesis is justified and conclusions drawn 
from the tests are "reasonable. ""'" 

Most researchers are familiar with basic design 
strategies and diagnostic tecfiniques for protection against 
blatant violations of these assumptions,, given that the 



model is reasonable ^ If possit^le, samples are drawn ran- 

domly, and assignments to groups are made randomly; outliers 

2 1 ■ • 

are screened, scatterplots are| examined; and variables 

are transformed as appropriate.! When necessary, to accommo- 
date knovsm violations of these assumptions — such as hetero- 
scedasticity (conditional "error" variance itself varying 
according to the values'""' taken on by other variables in the 
model) or serial correlation among the "errors" — estimation 
techniques other than ordinary least squares are employed. 
Such techniques include weighted and/or generalized least 
squares (Theil, 1971) as well as robust and resistant 
estimation (e.g., biweighted least squares; Mosteller and 
Tukey, 1977). No practical "tricks-of-the-trade , " however, 
can overcome substantive insensitivity to the model employed, 
particularly in its assumptions about the process whereby 
the outcomes are generated . ^ 

• Consider an all-too-familiar instance: A single 
program's effect is to be assessed via contrasts between 
scores in the treatment groii^ (the program's clientele) and 
an available comparison group. In many such applications, a 
researcher or evaluator must use "naturally occurring" 
groups, for whatever reasons. Care may be taken to strive for 
a comparison sample that represents a population very much 
like that sampled for the program's clientele. Initially, 
then, a contrast in means may be thought to be sufficient 
for testing the program's impact. The formal model is: 

[2] = m + g^ + e^j, j = 1, 2 groups. 

The assumption that e^^ is distributed indepen- 
dently of gj is crucial: if some other variable (e.g. , race, 
SES, sex, education, cognitive skills, age) is related to, the 
outcome measure Y and is differentially distributed in the 
program and comparison group., .this assumption is invalidated 
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and the test, clearly, is of . little i^use . (It is im- 
portant to note, here, that "error" appears in quotes be- 
cause the term e. . contains not only random, unintelli- 
gible disturbances or deviations from expected values of 
Y^j given j, but also all variables systematically related 
to . that are not included in the model underlying the 
test. . This type of modeling error, the failure to include 
important variables that are correlated with the outcome 
and other independent variables in the model, is known as 
•'specification, error-.") 

Consider a hypothetical case. Imagine that the 
program in question concerns assertiveness training, and 
is implemented within a multinational corporation for a 
random sample of its mid-level executives in a large sub- 
sidiary. Because these staff interact frequently with one 
another, "treatment diffusion" could lessen the ability to 
detect an effect on assertiveness if a control group were 
drawn from the same population. For comparisdn purposes, 
therefore, a random sample is drawn from the mid-level, 
executives of a second large subsidiary, engaged in a 
business very much like that of the first but located in 
another state; the executives in question have little 
interaction with each other across subsidiary boundaries. 
After completing the training program, some kind of as- 
sertiveness measure is taken on all* executives in both 
groups. 

Further suppose that the first subsidiary is a 
relatively new company, however, and its mid-level execu- 
tives are relatively young, with an equal male-female 
balance, The second company (providing the comparison 
sample) is older, with older mid-level executives (e.g*, 
characterized by slower but not otherwise necessarily 
different career growth patterns), most of whom are male. 
Finally, suppose further that age is positively associated 
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with assertiveness in both samples, and that men are more ^ 
assertive, typically, than women. These conditions inval- 
idate a simple t-test for the effect of the training program: 
e. . includes, two variables (sex and age) that are-associ- 
ated with .both assertiveness and group '*asrsignment" r the 
estimated average level of assertiveness ' in the comparison 
group is not a "fair" (in this case, unbiased) estimate of 
the treatment group' s ^average assertiveness level in the 
absence of the training program. 

The situation described — both in general terms and 
in the specific example provided — illustrates the primary 
problem of the "nonequivalent control group" quasirexperimental 
design (Campbell and Stanley, 1963). Some kind of adjustment 
to the estimated treatment effect — the contrast in group means — 
is required to account for the bias due to initial nonequiva- 
lence of the two groups. The most common resolution of this 
problem is to employ an analysis of covariance rather than ^ a 
simple analysis of- variance (the t-test for a difference in 
group means is Nothing more than an analysis of variance 
with, two groups)."* The covariates implicated in -violat- 
ing the assumption that the- ANOVA "errors" are independent 
of group membership are entered into a multiple regression, 
"controlling" or "adjusting" for the covariates* influence 
while estimating the (now adjusted) contrast in group means. 
The formal model is that given in [1]. 

Next consider the substantive meaning of this 
expanded (ANCOVA) model. The regression coefficients are 
given as (that is, presumed to be) identical within each 
treatment condition. In the common evaluation application, 
with one treatment and one comparison group, the regression 
surfaces (or lines, if there is but one covariate) are 
presumed to be parallel. In other words,, the net average 
program effect is assumed to be constant across all values 
of the covariate; the regression slopes — rates of change 

B-5 I ' ^ ■ ^ 



gRLC 

0., . 



in the outcome measure per unit change in the correspond- 
ing covariate — are presumed to be unaffected by the program 
intervention. Frequently, this ANCOVA assumption is vio- 
lated: if an expected program effect is the alteration of 
some relationship between an outcome and one or more co- • 
variates, the "h9mogeneity of regression" assumption (that 
of parallel regression surfaces) is untenable. This leads 
to a subtle but devastating violation of the assumption of 
independence of "errors" and covariates; the estimated 
treatment effect is biased — again — and conclusions drawn 

from the test of "adjusted" mean contrasts provided by the 

5 

ANCOVA estimation are apt to be very misleading/ 

"Heterogeneity of regression" poses/ a number of 
important problems for estimating and testing treatment 
effects. An effect can be defined conditionally as the 
difference between regression sui^faces at any given set 
of covariate values. Presumably, such effects can be* , 
averaged over values of the covariates in which anyone is 
interested. If the regression surfaces intersect within 
this range, however, this averaging will include effects 
in both directions tending to understate the magnitude of 
effects in either direction- Furthermore, if the covar- 
ij^tes are distributed differentially in the two groups , 
any such averaging should be weighted to reflect the prac- 
tical importance of this differential distribution. 

A different aspect of the "heterogeneity of re- 
gression" problem concerns the intended effect of the inter- 
vention. . It may be that a program's goal .is to alter the 
regression surface, creating "heterogeneity of regression" 
if the program is successful. Consider the assertiveness 
program illustration mentioned above. One objective of 
such an intervention could be to increase assertiveness 
most for younger executives, attenuating the existing rela- 
tionship between assertiveness and age (presumably enabling 
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managers to reach intended productivity levels earlier in 
their careers Under these circumstances, the most ap- 
propriate test of the -program Vs effect might be the con- 
trast in regression slopes rather than any average of 
differences in the regression surfaces. 



/ 



When the research design is of the "nonequivalent^ 
control group*' variety, there are other serious problems for 
demonstrating a treatment effect. While not unique to 
situations with "heterogeneity of regression, " the problems 
of logical inference are complicated when such heterogeneity 
is present. Of particular concern here is the attribution 
of cause for any statistical effect found. Are the diffe^^rMS 
found (either between regression surfaces or regressiop^ . 
slopes) due to the intervention or to other, perhap-^" unmeasured 

7 

variables? ^ 

While a number of (often overlooked) problems of 
ANCOVA models have been reviewed, one more deserves speci- 
fic mention. This concerns developmental outcomes, where 
age is or is expected to be an important covariate (e.g.,. 
any developmental assessment of children). Cross-sectional 
modeling of any developmental phenomenon poses difficult 
problems for analysis and hypothesis testing- If age at 
testing is unknown and ,dif-f erentially distributed across 
groups, the estimate of treatmen~t effects will be biased, 
the direction of bias depending upon the differential age 
distribution. Recording a^ge at testing and using it as a 
covariate will not necessarily be the appropriate adjustment, 
however. The relationship of age with test scores may not 
be linear, and some transformation of the test scores may 
be required to linearize the relationship with age. Many 
developmental tests also exhibit some kind of variance 
heteroscedasticity with age; the "fan spread" phenomenon 
is a we 11 "-known example,- where the variability of tests 
scores increases with- the age of the children ^being tested. 
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Also, many developmental tests exhibit sp^called "floor" 
and/or "ceiling" effects (lower and//pr upper limits to the 
possible test scores, effective for anyone being tested at 
some point on the age* continuum) . Such effects require a 
nonlinear and hereroscedastic modeling of tests scores by 
age, although with restricted age samples a linear model 
may suffice if extrapolation beyond the sampled age range 
is not required. (For ^ more technical discussion of 
problems in modeling developmental phenomena, see Bryk and 

i 

Weisberg 1976b). Finally, developmental interventions,! 
typically, are not expected to yield constant effects a]: 
every age; the expected effect is greater with increasing 
age. That is, the treatment effect expected is an 
accelerated growth rate. If a simple increase in the 
regression slope (test score on age) is expected, the ANCOVA 
effect estimator is not g^ — a group level of scoring 
effect—but another regression parameter for a group-by-age 
interaction— *a gtroup rate of growth effect. 

In suiTunary, traditional ANCOVA testing tech- 
niques employed in "nonequivalent control group" designs 
(especially) employ very strong assumptions about the 
relationships between "errors" and other variables in the 
model, particularly the variable indexing the treatment/ 
control group distinction( s ) . Unless these assumptions 
make substantive sense relative to the outcome(s) intended 
and measured, and to the covariates included in ar excluded 
from the analytic model, ANCOVA tests are very likely to 
be biased, inappropriate, and misleading as far as policy 
implications drawn from the ANCOVA estimates, and tests are 
conceir|;)ed . Even with a true experimental design, the ANCOVA 
model must make sense, substantively, or run the risk of 
providing misleading, irrelevant, and/or inappropriate 
information. 



B-8 



B,2 Value-Added Models 

An important characteristic of most regression 
adjustment strategies (of which ANCOVA is the prototypical 
model) is that they attempt to "correct" estimates of treat-^ 
ment effects that are problematic (for whatever reason). An 
oversimplified but useful characterization of the value- 
added approach is that it attempts to model an "expected" 
outcome in the absence of intervention or treatment, and 
estimates the treatment effect from the differences ( ob- 
served less expected-in-the-absence-of-treatment ) found in 
the treatment group. Unlike other regression adjustment 
models? a value-added model will not contain an "effect" 
parameter that is estimated and tested routinely in a 
regression or ANCOVA fit ling. Under certain conditions, 
a value-added model may become a model that is, for all 
purposes, an J^COVA models \ 

Value-added models have found frequent applica- 
tion in estimating program impacts on developmental phe- 
nomena (Bryk and Weisberg, 1976a; Bryk, Strenio, and Weisberg, 
1980). In its simplest form, one developmental model 
is : ' 

[3] = a^ + ^i^i ®i '(^i^s testing) 

where the parameters of interest are no longer assumed to be 
constant across individuals. Both the intercept (a^) and 
the regression slope Tb^) are assumed to be random , sampled 
from distributions with means (m^, m^) and variances 
(S^, S^) respectively. If a. and b. are constant across 

individual? (that is, = = 0), then this becomes 

a o 

a familiar, simpler regression: 

Y.=a+bA.+e. 
1 11 
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similarly, if interest is centered around the mean intercept 
and mean slope, [3] can be rewritten in like form: 



where et = e . + (a . ^ m ) + (b. - m, )A. . 

Ilia 1 D 1 

The value«-added parameter is introduced simply as 
another term ( ) , 

Y. = a. + b.A. + V. + e. 
1 1 111 1 

where v. is assumed to be distributed with mean and variance 

(m , S ). Often, the parameter of interest is m^ , the average 
V V " 

v^ue added as a result of the intervention. 

Frequently, it is more realistic to assume that 
growth rates are. related to important background character- 
istics; the developmental and value added models become more 
complicated : 

C5] y. = a. + (b^X^. + ... + b^X^^.) A. + et, and 

C6] y. = ^. + (b^X^. + ... + b^^X^.) A. + V. + el 



where et consists of e. and another "error" in the growth rate 
1 1 f 

modeling. Notice that the covariate terms here are all 

interaction-with-age-at-testing variables . This results 

directly from the assumption that growth rates rather than 

8 

intercepts (level of scoring) vary with covariates. 

Value-added models are particularly useful in 
estimating intervention effects on development with a pre-- 
and posttest research design. Pretest data typically are 
used to estimate average growth curves; deviations from 
posttest scores predicted by the pretest fitting, then. 
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I consist of a number of "error" terms (relative to the 

I quantity of interest, and the value added by the treatment) 

I and the value added. Under appropriate assumptions (e.g., 

the modeled "errors" have zero mean, are iijidependent of 
I each other, and are independent of the variables fitted in 

the pretest analysis), the mean of these differences is 
I , an unbiased estimator of the average value added. 

(Although the value-added approach to estimating 
treatment effects has been used with pre- and posttest 

(designs, typically, it can be used in a posttest only 
design where the group assignments have been randomly made. . 
Equations [5] and [6] hold for the control and treatment / 
I groups/ respectively. Of importance here is the assump- 

' ' tion that the control and treatment random parameters 

I are sampled from identical populations; random assignment to 

groups lends strong credibility to this assumption. Both 
I -equations ca^ be rewritten in the form of equation lAji . The 

problem for estimating m^, the mean value added, is to use 

I the control group data to estimate the mean (or regression) 
parameters (m^, b^, k = 1, K). Once these estimates 

Eare available, an estimate of the quantity ( v^ + dp 
can be obtained for each individual in/fche treatment group; 



^ , under the 'assumptions about e* mentioned earlier, the 

I mean value of these estimates is an unbiased estimator of m 

When these conditions are met (samples drawn and assigned to 
I groups at r.andom, and the modeled "errors" independent of 

covariates and the value added), this statistic is easily 
■ tested, for any given null hypothesis: a simple t~test is 

justified, if the sample is of sufficient size to mitigate 
concerns about nonncTmality . For small samples, more robust 
tests of resistant' estimates may be more appropriate. 



I 



I 
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B.3 Models Employed in Bayley Analysis 



I The CFRP research design includes elements of ex- 

perimental and quasi-experimental design, Clearly, the pop- 
I ulations available at each of the six^ impact study sites 

ERIC ^"^^ So 



are "natural4.y occurring" populations; that is, assignment / 
of families to -'these sites is (for all practical purposes)/* 
uncontrolled. \ Within each site, however, a sample of 
eligible famili^^ was recruited duringt-thli summer 1978? 
from each pool, families were assigjied randomly to CFIiP ^ 
and control/comparison groups. A^/ of the i^'pring 1979 data 
collection/ attrition, while of s.4rious mj^<3nt€ude / did not 
appear to be differentially seledtive across control /compar- 
ison and CFRP groups (Nauta, et al . , 1980, Appendix A) . At- 
trition from spring 1.979 to the time of Bayley data collection 
was much lower than that experienced during the first pro- 
gram ^ye^r of, the e^valuation. At this poin^, the two groups 
within each site still seem to be sampled from identical 
initial populations. 



O^BP i-nt.ent~-eo-n^erning '-chilTi^d^veiopme^ *'clear: 
by working through and with children's families, CFRP in- 
volvement is expected to promote children's developmejit . 
All CFRP children are' expected to benefit. Nevertheless • 
the mechanism or form that this intervention b@n'e"f it - is / \ 
expected to take is not clear. Whether the benefit ex-/ 
pected consists of a uniform boost in developmental measures 
and skills or a differential boost -acGardijig-'--txr'lclT^ /and 
family background characteristics is uriknown, a pViori . 
Arguments can be made for either type of expected J)rogr:^ 
benefit, particularly when the relative age and SES^ homogeneity 
of the evaluation sample is considered. Models of both 
types (traditional ANCOVA and value-added) were employed in 
exploring program impacts on Bayley scores . 

The six sites still pose problems for analysis, 
however. In additi to the potential problem of site-by- 
CFRP interactions, it is quite possible that regression 
surfaces are not parallel in the separate sites. (This is 
a problem for any value-added modeling as well as for an ^ 
ANCOVA.) Although a variety of interactions with sites 



c^^h be modeled and tested,' it is inconceivable that all 
possible in^teractions with site could be considered in any 
useful way. Two separate sets of analyses were conducted 
for both types of modeling: global analyses (including data 
from all sites) and within-site analyses. Although wi thin- 
site analyses will have far less statistical power (due to 
small sample size), they can offer some protection against 
inadvertent gross misspecif ication of global models. 

Prior' to estimating (ind testing) "final" models 
of' impact, much exploration was undertaken: outliers were 
screened and many data transformations were employed. The 
natural logarithms of per capita income and household si^e 
were taken / for instance. Dummy variables were coded (-1, 
1) in order to m.ltigate, to the extent possible, problems 
of multicolline^i ity in estimating models with interactions 
between these dum/uy variables and continuous measures. Age 
at testing, when ^ also used in interaction with dummy vari- 
ables, was adjusted by subtracting an arbitrary constant, 
so that the interaction variables would be continuous. 
[With age coded in months, from 15.00 upward, an age-by- 
dummy inter^action with the dummy coded (-1,1) jft^ill be 
distributed clumjis/ one with values less 

than or equal to -15, the other with values greater than 
or equal to 15. Arbitrarily forcing ag'^ to span the value 
of zero remedies thiSc two-clump distribution problem.] 
Finally, the relationship between age at testing and Bayley 
scores was examined in tietail, primarily ai^^ng control/ 
comparison group children. Effectively, th4^ age span was 
from 15 to 23 months, inclusive ,\ Within this\ age range, 
the relationship between age/and t^^st scores appears to be 
linear; there is no strong /evidence'.of "floor" or "ceiling" 
effects. No attempted modeling of tl^at feiation^^ip im- 
proved it. (A quadratic/ modeling of test scores d^i age and 
age-squareds. was tried ;/the natural logarii^hm and sqtiare 
root trans forK\at ions /of the test scores. 'were also regressed 
on age alone, w'lth po impjrovement Therefore , a /linear 



relationship of test scores with age was assumed in the 
models , Cised, throughout* 

ANCOVAS: The ANCOVA began with regressions of 
test scores on age, dummy variables for site and group 
(CFRP, control/compariso^'i) , and site-by-group interactions 
dummy variables. (As noted in Chapter 3,> St. Petersburg 
was eliminated from the ANCOVAs and growth model estima- 
tion in the value-added analyses.) Outliers were located 
(again) by examination of residual scatterplots with age, 
and frequency listings within sites and groups. After a 
preliminary age and site model was fitted (there were no 
CFRP effects), other sets of possible covariates were 
considered. The first set consisted of SES, family, and 
child characteristics; the second included a number of 
attitudinal measures derived from earlier parent interviews 
(see Nauta, et al . , 1980). Finally, site and CFRP inter- 
actions with covariates v*.,re explored. Wi thin-site ANCOVAs 
considered only age, the first set of covariates mentioned 
above, a CFRP effect, and then a CFRP-by-age interaction- 

Value-added: An average growth, model in the 
absence of treatment was estimated first (using only control/ 
comparison group children) . Initially, scores were regressed 
on age, age-by-site interactions, and sit€ dummy variables^ 
and a preliminary growth model constructed. Next, two sets 
of variables were considered, one containing age-by-SES, 
-family, and -child characteristics, the second with age-by- 
attitudinai measures . Site-by-age-by-coyariate interactions 
were explored next. A final growth model was then estimated, 
and used in the CFRP group to predict Bayley scores. The 
difference between observed and predicted scores contains 
the individual value added estimate plus , "errors'* ; the mean 
difference estimates the average value added by CFRP partici- 
pation. Due to the relatively small control/comparison 
group sizes within sit^s, within-site growth modeling 
considered age at testing only. 




Notes 



There are a number of other issues to be considered in 
judging the practice! importance of conclusions drawn, 
from such tests, of course, in addition to consideration 
of the statistical isignif icance obtained. For instance: 
Were the treatments actually implemented? Does the ^ 
research • design (including sample sizes) have sufficient 
powjer to detect effects of any practica^l significance? 
Is the design too powerful, e.g., is (are) the effect(s) 
detected too small to be of any practical importance 
(even though significant)? 

Univariate and bivariate outliers are located easily, 
typically using visual aids of some kind (histograms, 
scatterplots ) . Multivariate outliers are particularly 
probleiaatic for statistical estimation and testing, but 
often very difficult to detect ' (^.ee Gnanadesikan, 1977; 
Hoaglin and Welsch, 1978). / 
- . J 
The 'value of random assignment to treatment and controj. 
groups, of course, is to lend strong credibility to the 
assumption that "errors • in the model are distributed 
independently of group assignment or membership. But 
even with random assignment, particulatly with small 
samples, "errors" may be distributed differentially in • 
the two groups . 

It should be noted that the historical and theoretical 
development of ANCOVA was based on its value in increasing 
statisti^kX power in randomized experiments, not on its 
use as an "adjustment" mechanism for eliminating bias 
due to initial nonequivalence of the groups. Indeed, 
'blind, unthinking, use of ANCOVA models has come in for 
serious and warranted criticism (Elashoff, 1969; Cronbach, 
et al, 1976). Nonetheless, the subtle, complex, and 
serious problems of initial group nonequivalence do not 
necessarily pose insurmountable problems for analysis 
(Rubin, 1974, 1977, and 1979). 

If group assignments were made randomly, or if 'the ^ 
covariate(s) is (are) distributed identically within the 
groups, an argument for the "reasonableness" of the 
ANCOVA effect estimator can be made. Simply, it is the 
"average" program effect across all values of the 
covariate. Jf the covariate(s) is (are) distributed 
differentially within the two groups, and this is true 
necessarily with any "nonequivalent control group" 
design, the ANCOVA estimator, fails even in this sense. 
Furthermore, where "heterogeneity of regression" is 
found— whether intended or not, and regardless of degree 
of initial group nonequivalence — the non-parallelism of 



B-15 



Notes (continued) 



5. (continued) 

regression surfaces will almost certainly have important 
policy implications • The effective use of limited 
resources in implementing a program that appears to 
affect people differentially requires selective focusing 
in the distribution and/or use of • those resources. 
Further "adjustments" in estimating treatment effects 
are required (Rubiri/ 1974 and 1977). [A special case of 
this "heterogeneity of regression" problem can be found 
in aptitude-by-treatment interaction (ATI ) applications 
in educational, research.] 

6. Applications of within-group regression slope models 
of treatment outcpmes can be found under the rubric 
of "multi-level " analyses in educational evaluation . 
Complex and subtle issues of statistical testing and 
causal attribution abound here (Bache, 1980) • 

If this confounding of unmeasured variables with group 
membership is serious or extensive, or if it is sus- 
pected or known but the confounding variables have not 
been measured, it is impossible to attribute cause to 
the program intervention. (Bache and Nauta, 1979/ 
provide a very clear example of just this problem in 
a Home Start Followup evaluation.) With moderate 
confounding, an understanding of the selection process 
which renders the groups nonequivalent , and if the 
relevant selection variables are measured, estimates 
of treatment: effects — conditional on. values of the 
covariate(s) — with reasonable attribution of cause to 
the intervention can be developed (Rubin, 1977) . 

8. This does not prohibit models where covariates affect 
level o£ scoring on a developmental test rather than 
rates of growth. In contrast to ANCOVA models, however, 
the interaction-with-age variables assume priority over 
simple covariates . 

9. Testing this estimate formally is not --so straightforward, 
and this estimator is inefficient (see Bryk,' Strenio, 
and Weisberg, 1980). 
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Appendix C 

IMPACT ANALYSES 

As noted in Chapter 3, two separate sets of 
analyses of CFRP impact on Bayley scores were conducted, 
employing standard ANCOVAs and a value-added approach. 
The theoretical assumptions behind each are elaborated in 
Appendix B. The analyses themselves' are described here. 

C. 1 Mental Score Analyses 

Analysis began with examination of the relation- 
ship between age and Mental Score. Scatterplots within 
each site-by-group classification were examined for outliers 
As discussed in Chapter 3, the data for the St, Petersburg 
control/comparison group were anomalous; instead of follow- 
ing the typical pattern of positive correlation between, agfe 
and Mentr" Score, the plot in this group was flat (r = - 01) 
Therefore, St. Petersburg was excluded from ANCOVA tests of 
CFRP effects and from the growth model estimation prior to 
making global value-added tests. (The St. Petersburg CFRP 
group was included, however/ in the estimation and testing 
of global value added.) Figure C-1 displays Mental Scores 
by site and group; the St. Petersburg anomaly is quite 
striking . 

w'ithJkn the age range tested, the age/Mental Score 
relationship seems to be linear. There is no evidence of 
floor or ceiling effects, or of curvilinearity in the 
developmental plots. A number of alternatives were examined 
Mental Score was regressed on age and ag& squared; the 
natural logarii::hm of Mental Score was taken— an appropriate 
transformation if the grc^wth curve is bowed downward, as 
would be expected wit;h floor effects, or with a lower 
asymptote — and tried as an alternative to the raw score. 
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Figure C~l 

Bayley Mental Scores by Site and Group 
(CFRP on Left in Each Pair) 
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A linear relationship between ^ge and Mental Score was 
assumed throughout the analyses reported below. 



ANCOVAs : Two simple ANCOVAs. {site, CFRP, with 
age as a covariate) were -e<^ducted . The first (Table C-1 ) 
included all sites except St. Petersburg, and showed no CFRP 
effect. In the second (Table C-2), botlr'^S^ Petersburg and 
New Havan were excluded, because of the apparent tester prob 
aems at' these two sftes (see Appendix A); again, there was 
no CPRP e'ffect. r 



Source 

Age 

Main: 



Table C-1 



AMCOVA of Mental Score 



df 



MS 



6489.06 



6489.060 198.30 



000. 



Site 


184.29 


4 


46.07 


1.41 


.232 


CFRP ^ 


22.87 


1 


22.87 


0.70 


.404 


Site X CFRP 


106.07 


4 


26.52 


0.81 


.502 


Residual 


8409.82 


257 


32.72, 






TOTAL 


15743 .14 


267 


58.96 







(b 



age 



= 1.80; this represents a growti!^^ rate of 1.8 
points on Mental Score per \month .of age) 



St. Petersburg excluded. 



Source 

Age 

Main: 



Table C-2 
ANCOVA of Mentai See 



SS 



df 



MS 



6041.45 




(b 



age 



= 2i23) 



bNew Haven and St. Petersburg excluded.. 



6041. 45\ . 210.53 .000 



• Site 


.167.48 


3 


55.83 


1. 


94 


.123 


CFRP 


V 34.64 


1 


34.64 ' 


1. 


21 


.273 


Site X CFRP 


106,. 01 


3 


35.34 


1. 


23 


.299 


Residual 


6427.96 


224 , 


28. 70 








TOTAL 


12806.32 


232 


55.20 









\ . ' . 

The site and^CFRP measures were recast as dummy 
variables for further exploration of ANCOVA models (New 
Haven ..included) . As expected,. no dummy variables for sites 
are significant, but there appears to be a small Salem-by-. 
age interaction: without considering other covariates, the 
estimated growth rate for all other sites is about 2.00 
points per month, while in Salem it seems to be about 2.64 
points per month. (A two-tailed test of the interaction 
Coefficient is significant at .0*6. ) ; A number of covariates 

N. 

were considered next, in four sets: 

M> Child characteristics — sex, race, whether the 
oldest child, and whether "at risk" due to 
circumstances at birth (low birth weight or 
other physical problems). 

• SES — the natural logarithm of per capita 
income, an income source construct, whether 
or not the child's mother was employed, and 
whether she had finished high school at CFRP 
entry. 

• Family characteristics-^the natural logarithm 
of household size, and whethei;- the child's 
mother had ever been married (formally or 
informally) . 

• Other "softer" variables (constructed from 
mother's self-reported items) — the extent of 
social interaction with friends, the extent of 
social interaction in organized groups (church, 
political: clubs r etc.), a measure of "hassled" 
feelings -(generally), an assessment of the 
numbers of "worried" and "pleasing" situations 
typically^ faced (intended to measure "negative" 
and "positive" reinforcement for parenting 
roles or activities) , and two measures of 
infant temperament (one i^ncluding primarily 
reports -of physical well-being — e.g., a health 
problem Requiring frequent at"f:ention, feeding 

< problems^ and sleeping problems — and the other 
- containing items of a more social nature — for 
instance'/ difficulty in comforting, and doesn't 
smile or seems disinterested in things going on 
around , him/her ) . 
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Finally, site-by«-covariate interactions were ex- 
plored..- (At no point during the models-building process were 
CFRP or CFRP-by-age in,teractions significant.) 

The model finally adopted includes age, the Salem- 

by-age interaction, two' covariates (whether the mother was 

ever married and extent of social interaction with friends), 

and five site-by-covariate interactions ( Jackson-by-whether 

mother employed; Las Vegas-by-whether mother employed and 

Las Vegas-by-log of per capita income; and New Haven-by-sex 

and New Haven-by -whether the child is the oldest in the family) 

The estimated model is summarized in Table C-3 . Very little 

explanatory power is added with the inclusion of all vari- 

2 

ables other than age: R (age alone) =. .44; with Salem-by- 
age, R^ = .45; in Table C-3, R^ = .46 (all R^ "adjusted'' 

Table C-3 

. Global ANCOVA Model of Mental Score^ 



Age^ 

Sal em-by-age 

Mother ever married 

Social interaction with friends 

rackson-by -mother employed 

jas Vegas-by-mother employed 

iLas Vegas-by-ln (per capita income)' 
* d 
!New Haven-by-child * s sex 

1' d 
i New Haven-by-whether oldest child 

Intercept^ 





s . e . 


b 

_£ 


2.05 


.19 


<.001 


.39 


.18 


.03 3 


1. 28 


.41 


.002 


.89 


.46 


.052 


.68 


.41 


.092 


-.77 


.41 


.062 


® -.82 


.57 


.151 


.66 


.38 


.078 


-'.71 


.41 


. 086 


120. 35 


R^=.46 





^St. Petersburg excluded; N ranges from 221 to 265 (pair- 
wise deletion used in estimating regression ) . 

b 

Excepting the test for age, all tests are two-tailed. 

^Age is age at testing, in months, minus 18 months; the 

intercept, therefore, is the predicted score at age 18 

months . ^ ^ ^ 

d *^ 
All dummy variables, are coded (-1,1). 

^The natural logarithm of per capita income ($K) was used. 
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for the number of covariates in the equation) . CFRP and CFRP- 
by-ag^ variables simply have no evident effects on Mental Score; 
correlations and partial correlations appear in Table C-4. 

The ANCOVA model presumes parallel regression 
surfaces ("homogeneity of regression") within each of the 
site-by-CFRP groups, excepting those violations detectable 
as interactions shovy^n in Table C-3. Given the number of 
site-by-covariate interactions detected, as well as the 
known differences between sites, separate wi thin-site ANCOVAs 
were explored. St., Petersburg was excluded, again. Excepting 
Salem, the models adopted .are very simple and similar (Table 
C-5). Age , as . expected, is the most important covariate in 
all sites — even though it does not "explain" much in New Haven. 
Whether the tested child is the oldest child (oldest children 
score higher) appears to be somewhat important in Las Vegas 
and Oklahoma City. The extent of the mother's social inter- 
action with friends enters in two sites (more interaction, 
higher scores), Oklahoma City and Salem. Three more co- 
variates are included in the Salem model: child's sex (girls 
score ^ higher ) , the income source construct (families who rely 
more on earned income have children who score higher) , and 
the "high risk"* measure (children at risk score lower). -In 
all cases, CFRP and the age-by-CFRP measure are unimportant; 
there is no evidence of a positive influence on Bayley Mental 
Scores . 



Table C-4 

a 



Correlations with Mental Score 



Age-by- 
CFRP CFRP 



Zero order — - . 02 - . 03 

Controlling age -.05 -.03 

Controlling age and Salem-by-age -.06 -.03 

Also controlling covariates . -.05 "'^^ 

Also controlling site-by-covariates -.06 -.00 



^St. Petersburg excluded; total N=265. 

^CFRP is.coded (-1,1); -1 is the control/ comparison group, 
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Table C-5 



Wi thin-Site ANCOVAs, Mental Score 
(standard errors in parentheses) 



Las 



New 



Jackson Vegas Haven 



Age 



Oldest child 



Social interaction 
with friends 



Sex 

Income source 
construct 

High risk 
Intercept^ 

Partial correlations 
with Mental Score 

CFRP 

CFRP-by-age 



1.69 
(0.41) 



2.51 0.86 
(0.45) (o'.52) 

3.12 
(2.01) . 



Oklahoma 
City 

1. 56 
(0.26) 

0. 71 
(0.56) 

1.01 
(0.64) 



122.17 
. 26 



-.10 
-.16 



119.98 123.68 119.88 
.47 .06 .47 



18 
11 



-.12 
-.37 



-.20 
-.10 



Salem 

2.99 
(0.27) 



1.82 
(0.80) 



-0.89 
(0.61) 

1.19 
(0.75) 



-1 . 05 
(0.81) 

119.99 
.68 



.23 
.08 



St. Petersburg excluded. 

^Age is age at testing less 18 months; the intercept is the 
predicted -score at age 18 months. 

^All dummy variables are coded (-1,1). 
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Value-added ; The first step in a value-added model- 
ing of Mental gcore was to estimate a growth model among 
control/ compa,ri son scores (St. Petersburg excluded). Table >• 
C-6 gives descriptive statistics on the variables included 
in the model chosen; Table C-7 contains the corresponding 
correlations; Table C-8 summarizes the model fitted. In 
settling upon this model, groups of 'variables were considered 
in the following order: age, site, and age-by-site inter- 
actions; age interactions with SES, family, and child 
characteristics; ^and agH "interactions with parental behavior 
and attitudinal measures. The widespread searching strategy 
leading up to adoption of this model capitalizes on sampling 



Table C-6 

Descriptive Statistics, Variables ^ 
in the Global Mental Score Growth Model 



jntal Score 
c 



Mean 
123.60 
0.11 



SD 
7.58 
2.32 



N 
128 
130 



Age interactions 

' ~ d 
Oklahoma City 

High risk^ 



0.01 
0.19 



Per capita income -0.02 
Ever married 0.46 
Black'^ -0.53 



Salem-by-oldest 
child'^ 



0.17 



2. 28 
2.32 

1. 70 
2.24 

2. 27 

2.32 



129 
129 
100 
127 
129 

129 



^Control/comparison sample only; St. Petersburg excluded. 

^Missing data primarily are due to missing covariate in- 
formation, although a few extreme age-by-covariate values 
were recoded as mis'sing. 

^Age is at testing less 18 months. 

Dummy covariates are coded (-1,1). . 

^The natural logarithm of per capita income ($K) was used. 
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Table C-7 

Correlations Among Variables in the Global Mental Score Growth Model 



/ 

/ 



^Age is at testing minus 18 months . 
^Dummy covariates are coded (-1,1). 



d 



Th 



e natural logarithm of per capita income ($K) was used 



o lU, 





Me ntal 
ScQte 


Age 


Oklahoma 
City ^ 


High^ 
risk 


Per capita 
income 


Ever ^ ^ 
married Black 


^ b 
Age 


.69 










Age interactions 












• 


c 

Oklahoma City 


— • ft / 












High risk^ 


-.40 


-.44 


.05 








. d 
Per capita income 


.37 


.64 


.57 


-.21 






Ever married 


.25 


.27 


-.12 


-.15 


.36 




Black^ 


-.16 


-.26 


.38 


.14 


-.25 


-.50 


Sal em-by-oldest 
child^ 


-.07 


-.06 


.38 


.08 


-.02 


.00 -.14 


^Control/ comparison 


group only. 


St. 


Petersburg 


excluded . 







Table C-8 
Global Mental Score Growth Model 







o . c . 


b 
—id. 




1 . 76 


.352 


< . 001 


Age interactions 

d 

Oklahoma City 


-1.68 


.411 


<.001 


d 

High risk 


-0 .72 


.268 


.009 


e 

Per capita income 


-1.47 


.460 


.002 


Ever married 


0.94 


.304 


.003 


Black^ 


0.99 


.326 


.003 


Salem^by-oldest 
child^ 


- . 69 


. 286 


.018 


c 

Intercept 


123.51 
'.55 






^Control/ comparison 


group only 


, St. Petersburg 


Excepting age, all 


tests are 


two-tailed 





^Age is age at testing minus 18 months; the intercept is 
the predicted score at 18 months • 

Dummy covariates are coded (-1,1). 

®The natural logarithm of per capita income {$K) was used. 

(or random) errors in selecting variables for inclusion, so 
any interpretation of the model must be viewed skeptically., 
Nonetheless, the model estimated seems reasonable. With all 
other variables "controlled/' and within the age range tested, 
in addition to the expected age regression (or average growth), 
it suggests : - 

- • The typical age regression (or growth 
rate) in Oklahoma is flatter, by just 
over three points per month, than for 
children at the other sites. 

• The typical age regression (or growth rate) 
among children a't risk is flatter, by about 
one and one-half points per month, than for 
other ^children. 
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"\ • As per capita income increases, the age 

regression (or growth rate) decreases. 

• The typical age regression (or growth rate) 
of children in families where the mother has 
never married is flatter, by just over two points 
per month, than for children in other families. 

• The typical age regression (or growth rate) of 
black children (in this sample) is steeper, by 
jiist under two points per month, than for non- 
black children. 

• The typical age regression (or growth rate) 
among oldest children in' Salem is steeper, by 
just under one and one^half points per month, 
than for other children . 

There is a strong caveat here: the regression co- 
efficients are "par1:;ial" coefficients — that is, the estimated 
effects with all other variables in the eqiuation held "constant," 
statistically. For many variablefe,^ this "controlling" or 
"holding constant" has little relevance to what actually 
happens, as a glance at the correlations in Table C-7 will 
show. Consider the age-by -rOklaho^a City and age-by-race 
effects: one boosts" the estimated growth rate, while the 
other depresses it, other things being equal. But the Oklahoma 
City sample is predominantly, although .not wholly, black; in 
actuality , ' then,' these two estimated "effects" tend to "cancel" 
each other. Also, consider the age-by-per capita income effect. 
In Table C-7, this variable has a fair-sized positive correla- 
tion with Mental Score (.36). Yet, with other variables 
"controlled," the "effect" is in the opposite direction. 
(Indeed, it is precisely the substantial correlations shown 
in Table C-7 that lead to interpretive difficulties. If two 
variables are relatively 'uncorrelated, then "controlling" for 
one does not alter an interpretation of what the other measures, 
in effect. When two variables are relatively highly correlated, 
however, it is difficult to imagine just what meaning can or 
should be attributed to either when the other is "controlled.") 



C-11 




Given the growth model estimated in the control/ 
comparison group, Mental; Scores for the CFRP group were pre- 
.dieted, and the ( obs,ef yed-less-predicted) differences taken. 
These estimate the value added by CiiRP participation, and 
the mean of these dif f erences ' is an unbiased estimator of 
the mean CFRP effect. The mean of these differences is 
-.248, indicating no global CFRP impact (SD = 7.421, 
N = 116; p_ >.50). A one-way ANCOVA by site fails to reject 
the null hypoth^esis of equal means by site. There are no 
outliers evident in the distribution of these differences; 
all but two fall within two anc. one-half standard deviations 



of zero (as 99% of any set of c 



bservations sampled from a 



normal distribution with zero m^^an should be), and these two 
are just outside that interval (one positive, one negative). 
Figure C-2 summarizes the distribution oif value-added estimates 
by site. Nowhere is there any iifidication of an importa\nt (»or 
statistically significant) CFRP ^ffect. ^ 

For four sites, value-added analyses were ij-epeated 
within site. St. Petersburg, a^ain, could not be us^d. New 
Havenfs sample of onl^ 12 control/comparison cases cannot 
support estimation of any growth mod&l. (Indeed, the utility 
of value-added testing within sites is limited severely by 
small sample sizes. '^) Summaries of the four growth models 
fitted are given in Table C-9. Value-added estimates were 
generated within the CFRP groups at these sites, and outliers 
vy^ere screened using an arbitrary criterion of outside two 
(estimated) standard deviations of the sample mean. Results 
are summarized in Table C-10. ^ 



\ 

\ 

\ 
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Figure C-2 

Value Added, Bayley Mental S.cores, by Site' 
(Global Growth Modeling) 




St. Petersburg 



Each **X" represents an outlier. 



ERIC 



Table C-9 



Within-Site Mental Score Growth Models 



Age^ / 

Intercept^ 

N 

r2 



Jack-son 

2.61 
.123.95 

lb 

.48 



Las 
Vegas 

' 2. 74 

123.45 

25 " 

.53 



Okla- 
homa 
City 

1.66 

122.48 

36 

.44 



Salem 
2.71 
123.73 
33 
.84 



^Con'trdl/comj-arisoh groups only, St. Petersburg and New 
Haven excluded; s^ee tWxt. Wi thin-site models include 
only age, that is, they do not model the average growth 
rate as a function of covariates. 

Age is age at testing minus 18 months; the intercept is 
the predicted score at. age 18 months. 



Table C-10 
Within-Site Value-Added Summaries 



Mean 
SD 

Median of ^ 
Walsh averages 

Sampl'e median 

N . 

N omitted- 



sJacicson 



-1.78 
5.08 

-1.69 
-1.30 
29 
1 



Las 
Vegas 



-0 .71 
4.71 

-0.69 
0.06 
29 
1 



r 



Okla- 
homa 
city 



-1 .86 
3.03 



-1.82 
-2.29 
23 

1 



Sal\em 



2.18 
4.51 

2.49 
3.03 
27 

1 



'The mean value-added in Salem isx^signif icant at .04 
(one-tailed, test, with simultaneSti^ control for Type I 
error rates across." four tests) . 



observations . 
all pos sible 



A Walsh average is the ' average of any pai^r of 
The median of Walsh averages is the median^f. 
Walsh averages; it is a more efficient e.s tifnatol^o^'a 
population median than is t-he sample median, 

^Outliers were screened, using a two-SD rule; il cSmitted 
. shows how many. . 
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V It appear^s^-t^ia^CFRP in Salem has a, statistically 
significant positive effectNqn Bayley mental scores: a 
one-tailed t-test, controlling the Type I error rate over 
four simultaneous , tests, reaches^ a significance level of 
less than .04. Nevertheless, this ''effect" must be viewed 
with considerable skepticism: the small-sample growth models 
can hardly be adequate for explaining mental development. 
Figure C-3* summarizes wi thin-site value-added^results . 

In summary/ with only a tenuous hint at posi— - 
tive CFRP impact in Salem, there is no evidence that CFRP 
involvement has enhanced the mental development of the focal 
children in the evaluation study families, as measured by 
the Bayley Scales of Mental Development* 
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Figure C-3 



9 



Value Added, Bayley ^^ntal Scores, by Site 
(Within-Site Gr^owth Modeling) 




Jackson 



Las Vegas 



Oklahoma City Salem 



11 



C.2 Physical Score Analyses 

The analysis began with exploration/ of item-by-item 
scoring patterns. ' As noted in Chapter 3, pnly ten items from 
the Physical Development Scale had adequate variation in pass/ 
fail rates to be useful for analysis; these items served as a 
basis for computation of Physiqal Score. This measure repre- 
sents the proportion or"i terns 'passed for each case with- valid 
values on eight or mor.e of these items. All analyses were per 
formed an this Physical Score. ^ 

ANCOVAs;. An .initial ANCOVA, with age as the only 
"coyar iate T showed no CFRP effects and no site-by-CFRP inter- 
action (Table C-1.1); only age and site effects were found. 
Site and CFRP were recast as dummy variables, ' and a h.umber 
of stepwise regressions were -examined , following the strategy 
outlined previously for analyzing the Bayley Mental Score. 
Two .site 'dummy variables are significant (Jackson and Las 
Vegas children score lower, typically,' than 'children at other 
sites), as are four covariates (whether the child's mother 
is employed, the child's birth weight, whether the child is 
at risk, and the natural logarithm of per capita income). ■ 
Unlike- the Mental Score ANCOVA, no site-by-covariate inter- 
actions contribute anything to the Physical Score model. 
The model estimated is summarized in Table C-12. Correla- 
tions of CFRP and CFRP-by-age with Physical Score are shown 
in Table C-13. As. in the Mental Score analysis, there is no 

evidence of CFRP impact. \ / 

' • . / 

H 

... / 
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s Table 
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Global 


ANCOVA, 


Physical 


a 

Score 




Source 


SS 


df 


MS 


F 




Age 


2.760 


1 


2.760 


66. 75 


.000 


Main: 
Site 
CFRP 


1.139 
0.038 


4 

1 


0. 285 
0. 038 


6.88 
0.92 


.000 
.339 


Site-by-CFRP 


0.023 


4 


0.006 


0. 14 


.969 


Residual 


10.214 


247 


0.041 






TOTAL 


14.784 

(b 

age 


257 . 
0 .045) 


0 . 058 







^St. Petersburg excluded. 

Table C-12 

b 

Global ANCOVA Model of physical Score ^ 
(coefficients and -their standard errors are in 10 ) 



c 





b 


s . e . 


E 


Age 


4.22 


0.60 


< . 001 


Jackson^ 


-8.67 


.1.91 


< . 001 


TT e 
Las Vegas 


-5.10 


1.77 


.008 


Mother employed 


4.84 


1.62 


.006 


Birth weight 


-2.60 


1.25 


.075 


e 

High risk 


-3.00 


1. 77 . 


. .180 


Per capita income'^ 


3.65 


2 . 45 


.273 


d 

Intercept 


59.67 


2 

R^=.33 





^St. Petersburg excluded. ^ 

^Excepting the age coefficient, all tests are two-tailed 

^Age;is age at testing mi^^os^S months; the intercept is 
the score predicted atrf^^elS' months. / 

^Dummy covariates are c^pded (-1,1). / , 

*^The natural logarithiti of Annual per capita income ($K) 
was used . ^ 
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Table 


C-13 




Correlations 


with Physical 


Score^ 




CFRP 


Age-by-C 


Zero order 


-.06 


- . 07 


Controlling age 


-..09 


-.08 


Also controlling 
site 


-.05 


-.09 


Also controlling 
covariates 


-.02 


-.11 



^St. P^ersburg excluded. 

• \ 
As noted above, this approach to statistical model- 
ing casts a vidde net in looking for potentially relevant co- • 
variates; hen^, any interpretation of the models estimated 
must be viewed Skeptical ly. In a very nontrivial sense, this 
approach to modeling capitalizes on the vagaries of sampling 
(and measurement ): error in attributing "importance" to rela- 
tionships discovered through comprehensive search. Neverthe- 
less, the models estimated suggest that a number of character- 
istics-can influence physical development (as measured by 
Physical Score) substantially: employment, for instance, is 
"worth" more than two months growth,' typically, in this sample.. 
[Since dummy variables are coded (-1,1), the "effect" is twice 
the regression coefficient. Also, note that per capita . income 
is in natural logarithm units (of thousands of dollars). A 
change from $1500 to $2000 annual per capita income is a change 
of about 0.29 units of income modeled, while a change from 
$2000 to $2500 is about 0.22.] Being "at risk" costs about J 
one and one-half months growth, in this sample. 

Again, separate ANCOVAs within sites were explored. 
Table C-14 summarizes the important findings — or lack thereof 
\Ln the case of CFRP impact tests. Evidently ; CFRP has no 
p^itive impact on Bay ley Physical Scores — indeed, the values 
of partial correlations in Jackson, New Haven, and Salem hint 
at a negative CFRP effect. 



C-19 lit 



Table C-14 

Within-Site ANCOVAs of Physical Score Summarized 

2 

(intercept and age coefficients are in 10 units) 



Jackson 



Las i 
Vegas 



New 
Haven 



Okla 
homa 



Partial correlations 
with Physical Score: 

CFRP - • 26 

I CFRP-by-age -.36 



09 

,12 



-.10 
-.23 



.03 
.02 



Salem 



Intercept 


35 


.84 


63 


.84 


52.49 


84 


.22 


77 


.02 


Age slope 


4 


.54- 




.89 


5.94 


3 


.02 


3 


.95 


( s.e. ) 


(1 


.85) 


(1 


62) 


(2.67) 


(1 


.37) 


, (1 


.14) 


r2 




.15 




,37 


.23 




.31 




.20 






43 




51 


30 




59 




63 



.09 
-.20 



^St. Petersburg omitted 
b 



The covariates included in each model varied by site; the 
age coefficient given here is a "partial*' slope, "controlling 
for those other covariates. 
^Effective Ns for hypothesis testing varied, due to the^ 
use of the pairwise* delet^don of missing data values < 
given here are total sam;^le sizes. 



The Ns 



Value-added ; The estimation of the value-added 
model was done in the control/comparison group, excluding 
the Sti Petersburg subsample. (As noted, peculiarities in 
the mental scores among this group are evident and inexplic- 
able.) Table C-15 gives descriptive^ statistics on the set 
of variables included in the growth model estimation; Table 
C-16 lists vtheir intercorrelations . The initial growth model 
is given in Table C-17. 
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Table C-15 

Descriptive Statistics, Variables in^the 
Global Physical Score Growth Model 





Mean 


SD 


.,b 
N 


Physical Score 


0.53 


0. 


24 


121 


Age 


0.11 


2. 


32 . 


130 


Jackson 


-0.73 


0. 


69 


132 


Las Vegas^ 


-0.61 


0. 


80 


132 


Age interactions 










Jackson 


-0.25 


2. 


31 


130 


^ j\. 


0.22, 


2. 


31 


130 


Jackson-by-peg ^ 
'capita income ' 


0.01 


1. 


70 


100 


Jackson-by- 
fin ishgd high 
school 


-0 .08 


2. 


29 


127 


New Haven-by- 
finishgd high 
school' 


-0.38 


2. 


26 


127 


Ok 1 ahoma -by-pg r ^ 
capita income ' 


0. 06 


1. 


69 


100 


Oklahoma -by- ^ ^ 
..household size ' 


-0 . 11 


3. 


56 


128 



^rnntrbl /comparison group only, St. Petersburg exclud ed . 

\ ' ■ . V,.. T 

^Outli^rs were screened, but most missing data represent 

missirifl rather than extreme information. Pairwise" deletion 

was usfed in estimating the model. 

"^Age is \age at testing minus 18 months. - 
'^Dummy covariates are coded (-1, 1). 

®The natuiral logarithms of per capita annual income ($K)/ 
and household size were used. / 
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Table C-16 


ma mm mm I 








a 

OorrelatioiTs Anong Variables in the Global Physical Score Growth Model 
















Age interactions 






Eliysical 
Score 


Age 


Jackson 


las 
Vegas 


Jackson- „ * 

C C C Q. 

Jackson Sex by-incone ' 


Jackson- New Haven 
by-H.S.^ by-H.S.^ 


Oklahoma- - 
by-inccme . 


Age 


.58 














Jackson^ 


-.23 


-.11 












Las Vegas^ 


-.16 


-.04 


-.20 










Age interactions 
















c 

Jackson 


-.34 ■ 


-.78 


-.05 


.07 








Sex 


-.17 


-.02 


-.09 


.08 


.01 






1 income 
to 


-.20 


-.43 


-.01 


.04 


.64 -.14 




< 


to 

Jackson- 
by-H.S.^ 


-.12 


-.27 


.12 


-.07 


.27 -.13 .33 






New Haven- 
by-H.S. 


-.25 


-.35 


.07 


-.03 


.35 -.19 .20 


.64 




CScLaho^gby- 

incone ' 


-.13 


-.50 


.03 


-.01 


.30 -.24 \ .37 


.20 .35 




Oklahona-by- 
house^Jd 
\ size ' 


-.36 


-.41 


.07 


-.01 


<• 

.22 - -.03 .22 


.07 .13 


.55 


















^Control/canparison group only, St. Petersburg excluded. 
Age is age at testing minus .18 itonths. 
IXminy covariates are coded (-1,1). 

The ilatural logarithms of per capita annual inccme ($K) and household size were used 
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Table C-17 

Global Physical Score Growth Model^ 
(coefficients and t^^^ir standard 
errors' are in 10 units) 





b 


. s . e . 


b 
£_ 


TV C 

Age 


9.47 


1.52 


<.001 


Jackson 


-4.95 


2.75 


.145 


Las . Vegas^ 


-4.09 ' 


2. 25 


.139 


Age interactions 








d 

Jackson 


5.70 


1.64 


.001 




-1.10 


0.81 


.350 


Jack^bn- ^ 
. by-in'come 


-4.11 


1.56 


.017 


Jackson- 
by-H.S 


2.28 


1.09 


.073 


New Haven- 
by-H.Sv 


-3.67 


1.15 


• OU o 


bklahoma-^ ,^ 
by- income ' 


6.48 


1.52 


<.001 


Oklahoma-by- 
house^o^d 
size ' 


-1.68 


0.61 


.011 


Intercept 


45.06 

^ ' i 

- . ',■ • 1 


R^=.52 

{ 





^Control/comparison group, St . Petersburg excluded. 

^Excepting the coefficient for ag^, all tests are two- 
tailed. 

^Age is age at testing minus . 18 months. 
^Dummy cova2;iates are coded | (--1 , 1 ) • 

^The age-by-sex interaction was significant at <.15 

until the various age-by-site-by-covariate interactions 
were entered . , 

^The natural logarithms of per capita annual ind'ome ($K) 
and household size were used. 
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There are a number of interesting points to be 
made, with the caveat made in discussing the Mental Score 
growth model. Site, effects for Jackson and Las -Vegas axe 
still evident, but the Jackson-by-age interaction acts to ' 
"offset" the .negative Jackson effect (that is, scores in 
Jackson, typically, are below those in other sites except 
Las Vegasr but the growth rate in Jackson is steeper). All 
other things "controlled," the estimated growth rate for 
boys is slower than tjhiat for girls. Increasing per capita 
income in Jackson seems to depress the growth rate, whereas 
in Oklahoma City per capita ihcome appears to enhance growth, 
all else "controlled." 'Whether the childVs mother has finished 
high school has the same flip-flop effect on growth rates in 
New Haven and Jackson, respectively. 

Given this growth model, value-added ^timates 
were derived by subtracting predicted from observed Physical 
Scores in the CFRP group. A negative value was obtained; 
-0.019, standard error = .019, £ > .84. A one-way analysis 
of variance of these differences showed no differences by 
site; a breakdown by site is' presented in Table C-18. It 
appears that CFRP involvement has had no positive average 
effect on physical development as measured by Physical 
Score . ' 





Table C- 






Value- 


- Added Summaries 


, Physical 


a 

Score. 




Mean 


SD 7 ,• 


\^ N 


Jackson 


■-.015 


2-7-9^ 


22 


Las Vegas 


-.020 


.191 


28 


New Haven 


-.023 


. 235 


18 


Oklahoma 


-.03^ 


/.200 


16 


St. Petersburg 


-.075 


/ .159 


24 


Salem 


-.043 


.246, 


26 



^These are global value-added estimates broken down by site. 
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As with the ANCOVAs, separate analyses were con- 
ducted within sites; New Haven was dtropped from these analyses 
due to small samples. The growth models estimated are given 
in Table C-19. 

Value--added' estimates at these four sites are sum- 
marized in T.able C*-2lO. Again, thre^ of the four means are 
negative, as are three of the four medians. There is no 
evidence here of positive CFRP , effects on Physical Score 
values, and a suggestion of negative effects. (Note that 
the only positive estimate is in Salem, although it is not 
at all significant.) 
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Table C-19 



Within-Site Physical Score Growth Models^ 


(coefficients 


in 10 units) 








Okla- 






Las 


homa 




Jackson 


Vegas 


City 


Salem 


Age^ 10.08 , 


5.43 


3.39 


4.90 


Intercept 46.92 


41.3ri 


57o36 


57.49 


N'^:^ "^"-^--^ • 16 


23 


3e 


34 


.62 


.22 


.10 


.39 


^Control/comparison groups, 


St. Petersburg and 


New 


Haven excluded. 








Age. is age at testing minus 18 months. 






Table C-20 






Within-Site Value- 


Added Physical Score 


c 


( tabled entries, 


except N, are m 10 


units ) 






Okla- 






Las 


homa 




Jackson 


Vegas 


City 


Salem 


Mdan -12.8 


-2.3 


-3.6 


1.74 


SD 22.2 


15.8 


17.2 


22.2 


Median of 








Walsh ^ 








averages -12.0 


-1.9 


-3.9 


2.5 


26 


29 


j 22 


29 


e 

N omitted 1 


0 


1 


1 


*^St. .Petersburg and New Haven excluded 


• 





^A Walsh average is the ayerage of any pair of observations. 
The median of Walsh averages is the median of all possible 
Walsh averages; it is a more efficient estimator of a 
population median than the sample median. 

^Outliers were screened. N is tested sample size; N omitted 
shows how many outliers were screened. 
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' ^ Appendix D 

PROGRAM PARTICIPATION AND DEVELOPMENT 

D.l Measuring Participation 

Program— pa-rticipation is described generally 
elsewhere in "this report (Chapter 2)\ These measures 
consist of counts— -of the number bf home visits, number 
of parent education sessions, and number of infant-toddler 
sessions attended. (For purposes of exploring relation- 
ships between participation and Bayley scores, participa- 
tion counts continued' only to the date of te'sting.) The 
sample size is* 151, with participation data missing for five 
families. (As reported in Chapter 2, New Haven 'was excluded 
from all analyses of program pai:ticipation. ) The number of 
home visits ranged from no visits (two families) to 53 
visits (one f amily ) , . with a median of 14 visits. Twenty- 
ifive percent of the sample had 8 or fewer visits; twenty- 
five percent had 23 or more visits. In contrast, 65 families 
had no reported attendance at infant-toddler (I/T) sessions, 
and 78 families had no reported attendance at parent educa- 
tion sessions. The median number of sessions attended is 
4 parent' education and 6 l/T; the 25th and. 75th percen- 
tiles are (1, 9) and (3, 11), respectively. 

Average participation rates. were obtained simply 
by dividing the total participation measures by the number 
of months over which that form of participation occurred; 
'the length of time varies, but for most families -the time 
span is either 13 or 18 months. Average home visit rates 
ranged from .0.0 to 3 . 9 per month, with a .mean of 1.4 per 
month (median is 1.1), with lower and upper quartiles at 0.8 
and 1.7, respectively. Among those who attended parent 
education sessions (N = 66), participation ranged from 0.2 - 
to 1.7 per month, with a median of 1 . 0 and lower and upper 
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quartiles at (0.6, 1.^)*). l/T session participation rates 
1[N =^ 68) ranged from 0 . 1 to 4.5 sessions per month, with a 
median of 1.0 and quartile "hinges" at (0.8,. 1.6). 

Table D-1 lists selected inter-item correlations. 
Clearly^ total and rate of home visit participation are 
highly confounded; only the rate. was used analytically. 
Also/ total I/T and parent education session attendance 
are fairly highly correlated, so a composite total center 
participation measure was created; because it was highly 
skewed, its natural logarithmic transformation was taken. 
Four continuous participation measures', then, were used: 
home visit, I/T session, and parent ejducation session rates, 
and the natural log of a total center participation construct, 
Figures D-1 through D-7 give stem-and-leaf displays of and 
descriptive statistics for these measures. (These are very 
much like histograms, and can be read in much the same 
way. Each data point is represented; the value of any 
point may be read by taking the stem value and adding the 
leaf decimal, which is rounded. The two largest values in 
Figure D-l, for example, are 3.8 and 3.9.) 

Table D-1 

Spearman Correlations, Participation Measures 



Total Amount of 

Participation Measures Home Visits I/t Sessions 



a 



I/T sessions^ ^ -55 

Parent ed . sessions .40 .76 

Rate of Participation 
Measures ^ 

I/T sessions ^ * .39 

Parent ed . sessions .10 .05 

Home visits, total 

with rate • 86 

I/T sessions, total 

with rate^ • 44 

Parent ed. sessions, 

total with rate .06 



^These yariables^ are defined only for those families with 
some /appropriate center participation. 



Figure D-1 

Stem-and Leaf Display, Average Number 
of Home Visits Per Month 

(a for leaf 0, 1; b for leaf 2, 3'; c for leaf 4, 5; 
d for leaf 6, 7; e for leaf 8, 9) 
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^IR is the interquartile range; in a normally distributed 
population, .75IR is about one standard deviation. 
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Figure D-2 

Stem-and Leaf Display, Average Number 
of 'I/T Sessions Per. Month 



(a for leaf .00-. 24; b for leaf .25-. 49; c for leaf .50-. 74; 

d for leaf .75-. 99) 
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Values greater than 2.5 were receded as missing for 
analytic purposes. 
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Figure D-3 



Stem-and Leaf Display, Average Number 
of Parent Education Sessions Per Month 



for leaf 0, 1; b for leaf 2, 3; c for leaf 4, 
d for leaf 6, 7; e is leaf 8, 9) 
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Figure D-4 

Stem--and Leaf Display^ 
Number of I/T Sessions 
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Figure D-5 



Stem-and Leaf Display, 
Numt)er of Parent Education Sessions^ 
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Under certain conditions, counts of 0.5 were permitted. 
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Figure D-5 

Stem-arid Leaf Display, 
Number of Center Sessions, I/T and Parent Averaged 
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^If a family was missing data on only one of the two total 
participation measures averaged here, the value for the 
other was permitted to stand. 
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Figure D-7 • 

Stem-and Leaf Display, 
Natural Log of Number of Center Sessions 



(a for leaf ,0, 1; b for leaf 2, 3; c for leaf 4, 5 
d for leaf 6, 7; e for leaf 8, 9) 
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D.2 Measuring Family Needs and Strengths 

Program participation as well as developmental 
scores may be related to various family needs or strengths. 
Much of the data available concerning needs and strengths • 
comes from CFRP staff reports / or from interviews conducted 
only with CFRP mothers? the modeling of developmental scores 
(Appendix C) could not take these measures into account, 
since no control/comparison group data are available. In 
attempting to understand relationships between pa.i/ticipation 
in CFRP 'and developmental outcomes, however/ they/ must be 

considered. / 

I 

A total of nine needs or strengths variable's were 
identified or constructed. All are family-level measures. 
Some derive from parents* self -reports , others' from staff 
reports? some originate in data collected during the fall 
1978 (baseldne) field effort, others in the spring 1979 
' data collection . 

Two measures assess needs in a general way. One 
(HASSLED) is from the baseline parent interview, where a 
series of items asked how frequently the respondent felt 
"hassled" or bothered by various types of people (neighbors, 
doctor, friends, family, and relatives): HASSLED combines 
* these items, resulting in a single, self -reported measure of 
the frequency with which one feels , "hassled" generally (the 
response scale for each item ranged, in four steps, from 
"almost never" to "weekly," but the HASSLED scale retains 
only the direction of the item responses, from lesser to 
greater frequency). The second measure may be qonsidered to 
be a need index (NEEDS), and is based on staff reports 
collected on three instruments, one in the fall 1978 and two 
in the spring 1979 data collections. All items were of a 
checklist form, each item asking about specific needs (in 
areas of housing, health, income,, family management, etc.); 
subsequent analyses of these/ items failed to produce a set 




of relatively independent/ conten'. - based measures of need, 
so they were combined into one index. The NEEDS scale has 
no inlierent metric; a higher score indicates greater need. 
Figures D-8 and D-9 give steh^and-leaf displays for HASSLED 
and NEEDS. j 

Three measures assess family strengths with 
respect to CFRP participation. The first (CFRP SUPPORT) 
combines items from the fall 1978 and spring 1979 parent 
interviews. Two items in ,each interview asked about 
the degree of support for CFRP involvement from family ' 
and friends; these were combined into one measure, with a 
higher score indicating greater support. The second measure 
(RESOURCES) is an assessment of personal or social resources 
available to the family, and comes from the spring 1979 
CFRP staff interviews. It includes items considering the 
frequency of the mother's social , contacts outside of CFRP, 
her ties with an extended' family, and her awareness of 
social services available in the community. The , third 
measure of strengths (ENTHUSIASM) addresses enthusiasm for 
the program, and also derives from items in the spring 1979 
staff interviews. Staff were asked to rate mothers' degree 
of interest or enthusiasm for various CFRP activities; 
separate constructs by types of activities were impossible 
to distinguish, so one measure was created combining these 
items. During recent (spring 1980) interviews with staff 
from all eleven CFRPs, the concept of individual motiva- 
tion — although phrased differently by different staff — was 
mentioned universally as the single most important, ingredient 
of success in working with families ; ENTHUSIASM seems to be 
the only measure available as a motivation surrogate. These 
variables are described in Figures' D-10 through D-12. 

The remaining four variables are ambiguous in their 
valence: it is not clear whether a high score is more desirable 
than a lower score. The first three all have to do with the 
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Figure D-8 
Stem-and Leaf Display, HASSLED^ 
(stem units are tenths) 
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higher score indicates greater reported frequency of 
feeling "hassled" by others . 
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Figure D-9 
Stem-and Leaf Display, NEEDS^ 

(a for leaf 0, 1; b for leaf 2, 3; c for leaf 4, 5; 
d for leaf 6, 7; e for leaf 8, 9) 
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Figure D-10 
Stem-and Leaf Display, CFRP SUPPORT^ 

(a for leaf 0,1; b for leaf 2,3; c for leaf 4,5; 
\d for leaf 6,7; e for leaf 8,9) 
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Figure D-11 
Stem-and Leaf Display, RESOURCES^ 
(stem units are tenths) 



N 






/ £i \ 

\ o ; 


2 


c c c c c c 
p D D D D O 


V u ; 






\ 0 ) 


4 




(16) 




UUUUUUUUUUUUUUUU 


( . 3 ) 


6 


"7 "7 "7 

777 


V 30 ) 


7 


55555555555555555555555 5555555 


( 0) 


8 




\ 0). 


«- 9 




I 34 ; 


lU 


UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU 


( 0') 






(24) 


12 


5555555555555555 5555555 5 


( 5 ) 


13 


o o o o o 


( 0) 


14 , 




i 7') 


15 


0000000 


( 1) 


16 


7 


N=126 






Mean = 


.93 


Median =1.00 SD = .325 


IR 


. 500 


.75IR = .375 Skewness : =—.109 



(kurtosis -3.) . = . 586 




a 

A higher score implies greater personal resources ava,ilable. 



I 
I 



D.15 ^^'^ 



Figure D-12 
Stem-and Leaf Display, ENTHUSIASM 
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(a for leaf unit 0,1; b for leaf unit 2,3; c for leaf unit 4,5; 
d for leaf unit 6,7; e is leaf unit 8,9) 
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degree of help (from various sources) the mother has in 

ealing with the routine tasks of child care; these come 
from the fall 1978 parent interview. The first, called 
SPOUSE HELPS, assesses the extent to which the mother's 
spouse (or live-in male partner) helps. The second assesses 
help from other relatives (RELATIVE HELPS). The third 
combined responses concerning older children, neighbors, 
and babysitters (OTHERS HELP). It is not clear, prima 
facie, whether help with more tasks in any category indi- 
cates a positive resource — the availability of someone to 
rely upon for support — or a negative condition — that the 
mother gets additional help in whatever form because she 
cannot or does not sufficiently manage these routine tasks - 
for herself. There was a relatively high incidence of 
missing data on the series of HELP items; the constructs 
are summarized in Figures D-13 through D-15. 

The "fourth measure comes from the spring 1979 
staff interviews, and assesses the extent to which child 
development content was emphasized in working with the 
family during the initial (fall 1978 to spring 1979) phase . 
of jCFRP involvement (GD EMPHASIS). Again, a high score 
could represent either a positive or negative phenomenon: a 
family's needs may have been met sufficiently t6 devote 
relatively more time to child development activities, or the 
family's needs in the area of child development (nutrition 
and basic physical care) may have been so severe as to 
warrant relatively extensive attention. This variable has 
only three response categories; 58 families scored zero, 
minimum emphasis; 55 families scored one; 22 families scored 
two, maximum emphasis. 

Table D-2 shows correlations among the nine needs/ 
strengths variables. While these variables are not indepen- 
dent, quite clearly they represent a multidimensional set of 
measures. No, further reduction of this set of measures seems- 
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Figure D-13i 
Stem-and Leaf Displaif, SPOUSE HELPS' 
(stem units are tLenths ) 
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Figure D-14 
Stom-and Leaf Display/ RELATIVE HELPS^ 
(stem units are tenths) 
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Figure D-15 
Stem-and Leaf Display, OTHERS HELP^ 



(a for leaf 0,1; b for leaf 2,3; c for leaf 4,5; 
d for leaf 6,7; e for leaf 8,9) 
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Table D-2 

Correlations Among Needs /Strengths Variables' 



' CFRP RE- ENTHU- SPOUSE RELATIVE OTHER; 

HASSLED NEEDS SUPPORT SOURCES SIASM HELPS HELPS HELP 
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^Product-moment correlations. 



^The natural logarithm of OTHERS HELP was used. 
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warranted, although . CFRP SUPPORT, NEEDS, ENTHUSIASM, and OTHERS 
HELP seem to be moderately (positively) intercorrelated . 

Table D-3 lists correlations between the needs/ 
strengths set of variables and the five basic participation 
measures constructed • Clearly, any understanding of partici- 
pation in CFRP — a topic in the process/treatment study, to 
be addressed in a later report — must consider family needs 
and strengths • There are suggestions, and they cannot be 
followed-up here sufficiently, that: the fewer personal 
resources available the more home visits received; support 
for CFRP involvement may increase rates of participation in 
center sessions as well as the number of center sessions 
attended, but not whether center activities are avoided 
altogether; center session attendance in part may substitute 
for assistance from others in caring for children; and staff 
emphasis on child development in working with mothers may be 
a reaction to relatively infrequent attendance at I/T sessions. 

Table D-3 
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'^Product-moment correlations. ■ 

^The natural logarithms of total. center participation and 
OTHERS HELP were used. 
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■ D.3 Program Participation and Developmental Benefits 

y For two" major reasons, the exploration of rela- 

tionships between participation in CFRP and developmental 
benefits must be* done within the CFRP group only. First, 
participation measures are available only in the CFRP group. 

Given that the control/comparison group families do not 

J 

participate in CFRP, it might seem reasonable at first 
glance to impute zero- values for all participation measures 
\ and to include them in all analyses. Given the evaluation 

design, and indeed the CFRP demonstration intent, this would 
be wrong. CFRP is not ^necessarily intended to be the only 
developmentally beneficial program available to eligible 
parents in the demonstration communities; day care, develop- 
m^ental screening, and health and nutrition services are 
available to control/comparison families. The analyses 
reported in Appendix C addressed the question of whether 
CFRP involvement produces developmental gains (as measured, 
by the Bayley Scales) relative to whatever services are . 
available to and used by other families in the community. 
To impute zero values for participation to control/compar- 
ison families A^ithout otherwise "adjusting" for any relevant 
services they may be receiving would attenuat^e estimates 
of relationships of participation with developmental gains 
and obscure anything we might otherwise learn . about CFRP 
benefits to children. 

The second reason for limiting these explorations 
to CFRP families is simply "'that we have no comparable needs 
or strengths measures on control/comparison families. 
Inasmuch as program participation is a complex phenomenon, 
requiring some way to include measures of needs and/or 
strengths in understanding the relationship between partici- 
pation and developmental benefits, the control/comparison 
families will be excluded de facto. The exploratory analyses 
reported here, therefore, are restricted to CFRP families. 
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Finally, except for analysesXpf within-Salem 
Mental Score value-added estimates, participation analyses 
were "done globally, across all sites. TKte total sample 
available is 151, and with 95 to 146 valid \scores on home 
visits and developmental scores, leaving very small samples 
for simple analyses within sites. When centelj; participation 
measures are considered, the total sample is l^s than 70;' 
with certain interactions, it drops to about 30 .\ Analyses 
employing these variables cannot be done usefully Vithin 
sites. \ 

The dependent variables used are residuals and 
child-level value-added estimates from the developmental 
models estimated and reported in Appendix C. There are five 
variables: global ANCOVA Mental and Physical Score residuals, 
global Mental and Physical Score value-added growth model 
estimates, and within-Salem Mental Score value-added growth 
model estimates. Table D-4 gives descriptive statistics 
for these measures; Table D-5 contains their correlation 
matrix. Clearly, the two Physical Score measures cannot be 
expected to provide results differing by the growth model . 
adopted (ANCOVA or va^ue-added) : their correlation is -.87. 
The within-Salem Mental Scores, too, are highly correlated 
with their global counterparts: correlations are .82 with 
the global value-added estimates and .91 with the global 
ANCOVA residuals. The two global Mental Score measures are 
only moderately correlated (.68), and therefore they may 
yield different results according to the global growth model 
adopted. The global Physical Score measures are correlated 
only minimally with the global Mental Score measure, leaving 
the possibility that participation may be related to Physical 
and Mental Score measures in different ways. 

Participation Models 

There is no theory of just how program participa- 
tion ought to be related to developmental gains, and a fair 

•• ■ D-24 



Table D--4 



Developmental Outcomes for Participation Exploration 

Variable (N) Median Mean SD Skewness (Kurtosis 

Mental Score -0.23 -0.49 5.64 -0.09 -0.87 

Value-Added, 

or VADDGM^ 

(95) ' , 

ANCOVA Mental 0.16 0.37 4.52 0.08 ^ -0.58 

Score Residual, 
or ANCOVAGM^ 
(102) 

Within-Salem 3.03 2.18 4.51 -0.70 -0.12 

Mental Score 
Value-Added , 
or VADDED.M 

(17) • 

Physical Score -0.015 -0.019 0.197 0.049 -0.889 
Value-Added, 
or VADDGP 
(110) 

ANCOVA Physical -0.030 -0.018 0.196 -0.025 -0.409 
Score Residual, 
or ANCOVAGP 
(102) 



^The scale is that of the Mental Score analyses reported 
in Section- 3.2 — that is,, points on the Bayley Mental Score 

^The scale is that of the Physical Score analyses reported 
in Section 3.3 — that is, points on the Bayley Physical 
Score. (One-tenth of a point is one test item.) 

Table D-5 

9 

a 

Correlations Among Developmental Outcomes 

VADDGM ANCOVAGM \ VADDED^M ^ VADDGP 

ANCOVAGM • 68 0 

VADDED.M -82 -91 

VADDGP .26 -20 .43 

ANCOVAGP .29 .19 .52 .87 



^Product-moment correlations . 
Correlations within Salem only. 
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number of plausible arguments can be made a priori . The 
simplest model conceptually is^ the easiest to investigate 
empirically, but it suffers from obvious flaws. This modeX 
posits greater participation leading to greater developmental 
gains ("more is better"). This is not totally implausible: 
if some proportion of "activity time" (be it home visits or 
center sessions) is devoted to working with parents and 
child together (say, to demonstrate age-appropriate, develop- 
mentally stimulating activities), and if parents' effective- 
ness in learning and using these activities is a simply 
function of how many ( not how often) demonstrations they 
have been included in, then a "more is better" hypothesis 
would be justified. A more reasonable alternative, however, 
suggests that the relative frequency of participation — the 
rate at which families/children participate — is important. 
Relatively regular interaction may be required for the 
intended benefit to "take." The simplest version of this 
model is a "more is better" variant: more frequent partici- 
pation rather than simply more participation should lead to 
greater developmental ^benefits . 

! Both versions of this "more is better" model are 
deficient, however, in that they fail to account for possible 
relationships between types of activities in CFRP. There 
are three such types of possible 'developmental importance — 
home visits (HV) , parent ^education (PED) sessions, and 
infant-toddler (I/T) sessions. (The distinction between PED 
and I/T sessions is somewhat vague, empirically. The idea 
was to count sessions involving children separately from 
those involving parents only. The distinction in practice 
is not that clear, and program staff often had difficulty in 
recording any given center ■ session attendance as dnly one of 
these two types. Nevertheless, some distinction is evident 
in the data.) The two versions of a "more is better" model 
simply suggest that more or more frequent participation in 
any of these activities would lead to greater developmental 
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gains. The CFRP philosophy suggests that home visits and 
center sessions ought to be integrated and/or complementary 
in their focus and in their intended effect. That is, the 
effects of ^participating in any given type of activity ought 
not to be independent of the effects due to participation 
in other activities. The model presumed (at least tacitly) 
is an interactive one: the benefits expected when a family 
participates more (or more frequently) in multiple kinds of 
CFRP activities should exceed the benefits expected from 
each kind of activity alone. These might be called "more or 
more frequently in combination is better" models. 

Thus far, four (not necessarily mutually contradictory 
or exclusive) simple models of the participation-developmental 
impact relationship have been identified. In ascending 
order of complexity, they are: 

• Increased participation increases develop- 
mental gams (or "more is better"). 

• Increased V^tes of participation increase ^ 
developmental gains (or "more frequently is 
better"). \ 

• Increased participation increases developmental 
gains, but increased participation in both 
center and home visit activities increases 
developmental gains even more ( or "more in 
combination is better"). 

• Increased rates of participation increase ^ 
developmental gains, but increased partici- 
pation rates in both center and home visit 
activities increase developmental gains 
even more (or "more frequently in combina- 
tion is better"). 

These models are "simple" only in the sense that 
they fail to take any account of family needs or strengths. 
Since needs and/or strengths quite possibly affect CFRP 
participation and developmental gains made through such 
participation, needs and strengths ought to be considered 
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when modeling any relationship between CFRP participation 
and developmental outcomes.- Again, we have no theory 
to specify how' needs and strengths should affect the 
relationship (or, for that matter, just which needs and 
strengths to measure). Three basic ways of including needs 
and strengths can be identified. First, it is clear that 
CFRP families vary in the type and degree of needs that they 
have, and that participation (whether total amounts, or 
rates of participation) may vary according to family needs. 
It is quite possible, therefore, that a standardization of 
participation measures relative to need might be useful the 
model here is that increased participation or participation 
rate per unit of need leads to greater developmental gains. 
In the CFRP evaluation data, only one general extent-of- 
needs variable could be distinguished (NEEDS), and a set of 
Wrticipation variables (total amounts and rates) were 
standardized relative to it. The resulting constructs were 
so \ighly correlated with their simpler, unstandardized 
counterparts that the standardized versions cannot be judged 
to cbntain any additional information. (in all but one 
instance, rank order correlations exceeded .85.) Needs- 
standardized measures were dropped from consideration. 

A second conception of how needs and strengths 
affect participation and developmental outcomes is that of 
the simple linear model: needs and/or strengths covary with 
both participation and developmental gains, are causally 
prir to both, and must be "controlled" or "adjusted for" in 
examining the relationship between participation and develop 
mental impact. The analytic strategy suggeated here is that 
of regression modeling of outcomes on needs and strengths 
first, adding participation variables later (with appropriat 
needs and/or strengths "controlled"). 

i . 

The third way in which needs or strengths vari- 
ables can fit into a model of the participation/development 
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relationship is interactive. The participation effect 
itself may vary with family needs and/or strengths. For 
instance, increased participation may lead to greater 
developmental gains for families with fewer needs or more 
strengths than for families with ^riore needs or fewer strength 
The appropriate independent variablW under this model are 
products (interactions) of participation measures with needs 
and strengths variables. 

All told, therefore, there are twelve sets of 
models potentially of some use in ex-ploring the relationship 
between participation and developmental gains: four "simple" 
models, as listed earlier; four corresponding models in 
which needs and strengths must be considered as covariates 
of developmental outcomes; and f our \corresponding models in 
which participation interactions with needs and strengths 
are explored. Without a strong theory from which to argue 
the inappropriateness of any model, all must be considered 
in these exploratory analyses. Furthermore, with sample 
sizes as small as those available, and given the problems of 
collin^ariby often faced in trying to estimate (and compare) 
main effects and interactions simultaneously in regression 
models, we cannot possibly hope to construct "critical 
tests" of these alternative models here. All we can do is 
look for indications that participation may effect develop- 
mental outcomes in a manner consistent with the predictions 
of any of these twelve models. Without a strong theory to 
'back any exclusions a priori , all must be considered. 

Results 

The "simple" models are easiest to explore, since 
they do not consider needs and strengths measures. The 
correlations between developmental outcome scores and total, 
participation, participation rates, and various participa- 
tiorij^^^teractions are given in Table D-6. (Examination of 
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Table p-6 ■ ' ) 

Correlations Between DevelopmentaEl ^ 
Scores and Various Participation Measures 

VADDCa^ ANCOVAGM VADDED.M^ VADDGP ANCOVAGP 

Home Visit Rate -.02 .06 -02 .11 .17 

I/T Session Rate .10 .39 .19 .32 .27 
Parent Ed. Session 

Rate -.05 .05 .20 -.07 -.16 
Center Participation 

Dummy^ .06 --.00 .25 .20 .13 

Total Center Participation^ .04 .14 .48 .02 .15 
Interactions . 



Home Visit Rate by 
Center Participation 
Dummy 



03 .04 ^ .25 .13 .16 



Home Visit Rate by Total 

Center Participation -.05 . 04 . .16 .03 .16 

Home Visit Rate by I/T 

Session Rate -.02 .41 .18 .13 .24 

Home Visit Rate by Parent 

Ed. Session Rate -.05 .05 .10 -.17 .01 

I/T Session Rate by Total 

Center Participation .02 .29 .15 .31 .26 

Parent Ed. Session Rate by 

Total Center Participation .01 -.06 .17 .02 -.12 



^Sample sizes vary, according to pairs of variables. In general, for 
global outcgmes with the home visit rate and center participation 
dummy variables, N is about 95; within Salem, about 25. For global 
outcomes with I/T session rate, parent education session rate, and total 
center participation, the sample size is about 60;. within Salem, approxi 
mately 15. Sample sizes 'for correlations involving interactions may be 
somewhat smaller. (For sarhple sizes of 25 and 15, correlations of .22 
and .29 are significant in one-tailed tests at the .15 level; for Ns of 
95 and 60, with one-tailed tests at the .05 level, the appropriate 
correlations are .17 and .21.) 

These are within-Salem^ only. 

^This is at dichptomous variable. 

^The natural logarithm of this variable was used. / 
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scatterplots revealed few outliers, which were screened 
before computing these correlations, and no evidence of 

nlinearity in these bivariate relationships.) Although 
there are no indicators of significant correlations between 
participation and Mental Score outcomes under the global 
value-added model, there are'^fe'ome positive signs for each of 
the other four outcome possibilities. 

The I/T session attendance rate is positively 
correlated with both versions of the Physical Score gains, 
and with the ANCOVA model Mental Score gains. Total center 
participation and perhaps the center participation dummy are 
associated with value-added model Mental Score gains in 
Salem. Among the participation interaction variables, two 
stand out: the interaction between home visit and I/T 
session rates is correlated fairly highly with the ANCOVA 
Mental and Physical score residuals ; and the interaction 
between I/T session attendance rate and total center partici- 
pation — in essence, greater frequency of attendance over 
longer periods of time — is positively correlated with ANCOVA 
Mental Score residuals and with bot.h versions of Physical 
Score gains. 

If regression models are estimated using the 
entire set of participation measures as potential covariates, 
the resulting models are quite parsimor^ous . Since the I/T 
session rate is highly correlated with the home visit 



rate-by-l/T session rate interaction (.75)\and with the I/T 
session rate-by-total center participation 92 ), models for 
the ANCOVA residuals cannot be estimated usefully with more 
than one of these three variables in the equation. Alterna- 
tive models for each outcome can be estimated, ohe with the 
I/T session rate and another with the appropriate i^nteraction 
term. Prom the regressions estimated, predictions 
typical gains^ given specific values of the participat\on 
variable can be made. Predicted gains for simple models 
(with I/T participation rate as the only predictor) are \ 
shown in Table D-7. 
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Table D-7 






Estimated Gains 


from i/T 


Sessions per 




Session Rate° 


ANCOVAGM^ 


ANCOVAGP 


VADDGP 


0.6 


-1.36 


-.070, 


-.081 


0.8 


-0.64 


- . 048 


-.055 


1.0 


0.07 


-.027 


-.030 


1.2 


• 0.79 


-.005 


-.004 


1 4 


1. 51 


.016 


.021 


1.6 


2.23 


.037 


.047 


1.8 


2.95 


.059 


.073 


2.0 


3.66 


.080 


.098 



^These are points (or number of items) on the Bayley 
Mental Score. 

^These are tenths of a point (or item) on the Bayley 
Physical Score. 

^These are average number of I/T sessions per month. 
The sample mean is about 1.1 I/T sessipns per month. 

If these models have any validity, it seems that 
the participation rate in I/T. sessions must reach about 
three sessions every two months to produce positive gains on 
the Physical Score, and should be at (or greater than) two 
sessions per month for these gains to be appreciable — ^"that 
is/ to approach an average increase of one full item on the ' 
10-item scale. For gains on Mental Scores based on the 
ANCOVA modeling, once per month seems to produce a very 
small positive gain, while the average participation rate 
should exceed three sessions every two months for appreciable 
Mental Score gains. (In the evaluation sample, an average 
CFRP gain of about two points at 18 months of age — approxi- 
mately the sample mean age — would put CFRP children at the 
national norm for that age. Of course, as they get older, 
gains relative to a comparable non-CFRP population would 
have to be greater to keep the children at the national 
norm. ) 
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From the within-Salem value-added Mental Score 
regression, attendance at roughly 12 center sessions to the 
date of Bayley testing .(combining both I/T and parent 
education sessions) prec^icts a typical Mental Score gain of 
about three points . Most of the children were -feissted after 
about one year in the program, so this amounts t6 only one 
center ' session per month, in the within-Salem model , (This 
relationship was estimated on a sample of only 22 children, 
however, and should not be given excessive weight.) 

The introduction of needs and strengths variables 
has little effect on the relationships estimated in the 
"simple" models above. Correlations between needs /strengths 
and developmental scores are given in Table D-8. Three 
general points stand out: 

I B 

m NEEDS, RESOURCES, and ENTHUSIASM are all posi- 
tively correlated with Mental Score outcomes, 
J but not with Physical Score outcomes. 

I m CD EMPHASIS is correlated negatively with 

Mental Score gains, suggesting (but only 
suggesting) that staff emphasis on child 
development activities may reflect perceived 
need in this area rather than strength in 
other areas and concomitant opportunity 
to move beyond more basic family concerns. 

o RELATIVE HELPS is negatively correlated with 
each of the developmental outcomes, suggesting 
that what was measured here is more of a develop 
mental liability than an asset. 

In regressions on Physical Score gains, with needs 
and strengths taking precedence over participation measures 
initially, the relationships between participation and 
Physical Score are unaffected. In the within-Salem analysis, 
the NEEDS variable enters first, but the total center 
participation measure is still strongly and positively 
associated with value-added Mental 'Score gains (the partial 
correlation, with NEEDS controlled, is about .50, up from 
•48). In the ANCOVA-based Mental Score gains, RESOURCES and 
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Table D-8 . ^ 
Correlations Between Needs /Strengths Measures 
and Developmental Gains^ 

VADDGM ANCOVAGM VADDED.M VADDGP ANCOVAGP 



HASSLED 


.09 


-.01 


-.11 


.01 


.00 


NEEDS 


.18 


. 07 


.33 


. 06 


. 01 


CFRP 
SUPPORT 


. 02 


. 10 ' 


. 20 


- . 02 


— . (jy 


RESOURCES 


.12 


.24 


.09 ■ 


.13 


.06 


ENTHUSIASM 


,.08 


.18 


.25 


.09 


.04 


SPOUSE 
HELPS^ 


-.03 


.02 


-.31 


.03 


-.14 


RELATIVE 
HELPS^ 


-.26 


-.19 


-.38 


-.22 


-.24 


OTHERS , 
HELP"^' 


-.07 


.03 


-.16 


.06 


-.06 


CD 

EMPHASIS 


-.21 


-.23 


-.08 


-.12 


-.03 



^The needs and strengths variables are described in Section 
D . 2 . 

V 

■I— 

Within-Salem only. 

^These variables had high proportions of missing data and 
were excluded from regression analyses. 

^The natural logarithm of OTHERS HELP was used. 

CD EMPHASIS both enter the regression, but I/T session 
attendance rate is still strongly related to the 'outcome 
measure (the partial correlation with RESOURCES and CD 
EMPHASIS controlled is .33, down from .39). 

The final set of models to be explored focus 
on interactions between participation and needs/strengths. 
With six needs and strengths variables (the HELP series was 
omitted due to large proportions of missing data) and five 



"simple" participation measures, there are 30 "simple" 
interactions to explore. From among this set, four bivariate 
relationships are suggestive, each for a different outcome 
variable with the interaction between l/T session rate and 
RESOURCES: with VADDGM , the correlation is -34; with ANCOVAGM / 
.41; with VADDGP, -43; and with ANCOVAGP, .34. These 
correlations, and the regression models estimated as a 
result of exploratory analyses, suggest that ' the impact of 
rate of attendance at l/T sessions varies with the personal 
or social resources available to the mother, so that develop- 
mental gains due to increased rate of l/T session attendance 
would be. greater among children whose mothers have relatively 
more resources available to them. The effect of this inter- 
action is shown in Tables D-9 and D-10. 

Table D-9 

Estimated Physical Score Value-Added, 
by RESOURCES and l/T Session Rate 



Average Number 
of I/T Sessions 
per Month 


Low 


RESOURCES 
Medium 


High 


0.50 


-.125 


-.095 


-.065 


0.75 


-.097 


-.052 


-.006 


1.00 


-.069 


-.009 


.05 2 


1.25 


-.041 


.034 


. 110 


1.50 


-.012 


.077 


.168 



a 



Table entries are tenths of a point (or item) on the 
Bayley Physical Score, a 10-item scale. 

^Low RESOURCES are one standard deviation below the sample 
mean; medium RESOURCES are at the sample mean; high RESOURCES 
are one standard deviation above the mean. 



15 



Table D-10 

Estimated Mental Score Gains from the ANCOVA Model, 
by RESOURCES and I/t Session Rates 



Average Number 
of I/T Sessions 
per Month. 


Low 


RESOURCES 
Medium 


High^ 


0,50, 


-1.93 


-1.27 


-0.62 


0.75 


-1.32 


-0.33 


0.65 


1.00 


-0. 71 


0.60 


1.91 


1.25 


-0.10 


1.54 


3.18 


1.50 


0.51 


2.48 


4.44 



^Table entries are points (or item) on the Bay ley Mental 
Score. 

^Low RESOURCES are one standard deviation below the sample 
mean; medium RESOURCES are at the sample mean; high RESOURCES 
are one standard deviation above the mean. 

In reading Tables D-9 and D-10, it is important 
to remember that the average rate of participation in I/T 
sessions is just over once per month. The distribution of 
sample families by^ RESOURCES and I/T participation rates is 
centered around the values in the middle of these tables; as 
values change in any direction, the predicted "gains" .are 
more unstable, less reliable, or less accurate. Nevertheless, 
if these models have some validity, it is clear that partici- 
pation rates in I/T sessions can influence developmental 
scores appreciably — especially if the home visiting activities 
are successful in . increasing the mother * s awareness of social 
services available in the community and in creating and/or 
strengthening the family *s ties to supportive social networks 
(these kinds of items make up the RESOURCES construct). An 
average gain of c-ven one-third to one-half of a point on the 
Bayley Physical Score is impressive (the scale used consists 
of ten items altogether); as mentioned earlier, an average 
gain of about two points on the Mental Score, at 18 months 
of age, would put the CFRP sample at the national norm. 



A final set of explorations was done, checking 
third-order interactions. These were home visiting rate by 
I/T session rate by 'each of three needs/strengths variables: 
NEEDS, CFRP SUPPORT, and RESOURCES. Frbm these analyses, 
only the global value-added Mental Score outcomes are 
related sensibly to the interaction with RESOURCES. (The 
sensibility of exploring interactions of this order is 
debatable, especially without a theory to suggest that they 
should take precedence over simpler expressions of needs, 
strengths, and participation. Nevertheless, thesfe particular 
interactions do have a plausible backing in the CFRP applica- 
tion.) This suggests that greater RESOURCES, more frequent 
home visits, and more frequent I/T session attendance each 
mutually reinforces or "levers" the impact of the others. 

Summary 

Despite inability to detect group differences 
in Bayley developmental scores (CFRP versus control/compar- 
ison children) after 12 to 18 months of enrollment, it 
appears that certain aspects of CFRP participation could be 
expected to produce demonstrable, important developmental 
gains under appropriate conditions. Increased participation 
in center sessions, particularly I/T sessions, evidently can 
enhance children's development. Depending on the partici- 
pation model chosen as most appropriate, center session 
participation at rates that are somewhat higher than those 
currently found — at least in the evaluation sample— and more 
widespread attendance at center sessions (approximately half 
of the evaluation families had never attended a center 
session as of the . Bayley testing) conceivably could produce, 
demonstrable impacts . 

Potential interactions with RESOURCES ^nd with 
home visiting rates further attest to the potential in the 
CFRP concept. From limited empirical evidence, it seems that 



an integrated, family-focused approach which employs a 
mixture of home visiting and center sessions could produc 
important developmental impacts among children in CFRP- 
eligible families at ages as young as 18 months. The 
sample of families in the CFRP evaluation, howeve^r, have 
so far not participated with anywhere near the frequency 
required to offer any hope of detecting such impacts.'' 
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